mlmm-evaluation icon indicating copy to clipboard operation
mlmm-evaluation copied to clipboard

Published 20 hours ago • verified publisher icon nlp-uoregon

Few Shot configuration

Open Nkluge-correa opened this issue 6 months ago • 0 comments

Hello!

Is there a way to control how many examples are used to evaluate the models? Also, how are the evaluations currently set up? Are all benchmarks (ARC, MMLU, HellaSwag) running in a zero-shot fashion? If not, what is the configuration used?

Nkluge-correa avatar Aug 09 '24 12:08 Nkluge-correa