evals icon indicating copy to clipboard operation
evals copied to clipboard

Published 20 hours ago • verified publisher icon openai

Code Evals

Open billxbf opened this issue 1 year ago • 0 comments

Describe the feature or improvement you're requesting

I wonder if anyone has a solid method for evaluating code benchmarks like APPS. String typed codes can be very noisy and require deliberate preprocessing to be executed and tested. I don't see any class inheriting Evals that can perform code tests.

Any clue? :thinking:

Additional context

No response

billxbf avatar Jul 03 '23 03:07 billxbf