lm-evaluation-harness icon indicating copy to clipboard operation
lm-evaluation-harness copied to clipboard

Evaluation fails when all samples are cached

Open baberabb opened this issue 6 months ago • 0 comments

When all samples are already cached, the process errors out instead (instead of skipping to the metric calculation) on the subsequent run due to lack of requests to pass on to the LM.

baberabb avatar Jul 27 '24 11:07 baberabb