mteb icon indicating copy to clipboard operation
mteb copied to clipboard

Allow Evaluation should return results from cached runs instead of skipping them

Open nprime496 opened this issue 6 months ago • 1 comments

The evaluation do not run tasks twice, that is smart as it saves ressources but why is the task deleted ??

I am using the python SDK and spent several minutes wondering why my code didn't work anymore (do not return any result) and that was the culprit.

I'd thought that the package should use already cached results instead of just not returning anything.

Any design justification ? Is it a bug ?

nprime496 avatar Jul 30 '24 16:07 nprime496