simple-evals

simple-evals copied to clipboard

Hi, Thanks so much for publishing eval details like these--it's really appreciated! I was wondering whether there are any intentions to publish the eval settings + prompts for the GPT-4o...

haileyschoelkopf

If the bot uses over x seconds without a respond, the bot should return that the answer could not be answered.

linnemilieTv2

Question: Why is Claude running with temperature 0 and GPT4 with temperature 0.5? wouldn't that be a handicap for Claude in Humaneval?

1

zurferr

Remove unused imports and variable. [Ruff](https://docs.astral.sh/ruff/) was used for detection a removal.

esadek