verifiers icon indicating copy to clipboard operation
verifiers copied to clipboard

Ability to skip rollout and load cached responses

Open mikasenghaas opened this issue 2 months ago • 1 comments

Would be super useful for debugging a rubric and unit testing if we one could bypass the rollout function and simply load past responses, probably from results.jsonl file in outputs.

mikasenghaas avatar Oct 07 '25 15:10 mikasenghaas

+1 on that. esp with different env args. In my case i have an env where the parser behaves differently to support thinking models, and i forgot one too many times to enable the think flag :'(

i.e.

# on load_environment
if use_think:
    parser = vf.ThinkParser()
else:
    parser = vf.Parser()

would be happy to open a PR if it makes sense.

ob1-s avatar Oct 09 '25 22:10 ob1-s