verifiers icon indicating copy to clipboard operation
verifiers copied to clipboard

Disentangle scaffold and dataset

Open mikasenghaas opened this issue 1 month ago • 1 comments

@daniel can add some color here as well, but the gist of it is not clear how to handle scaffolds/ datasets, e.g. for SWE environments we have

  • scaffolds: mini swe agent plus, deepswe
  • datasets: deepswe, swe bench verified, swe smith/ etc.

should there be one env per scaffold with configurable dataset? one env per dataset with configurable scaffold? one env per scaffold and dataset?

mikasenghaas avatar Nov 27 '25 21:11 mikasenghaas

for reference take a look at deepswe or mini_swe_agent_plus environments. or as discussed here

rasdani avatar Nov 28 '25 02:11 rasdani