lm-evaluation-harness
lm-evaluation-harness copied to clipboard
add Russian mmlu
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.
:white_check_mark: tatiana-iazykova
:x: Tatyana Yazykova
Tatyana Yazykova seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.
@haileyschoelkopf @lintangsutawika @baberabb do you mind to have a look at it? I just took existing mmlu code and adapted it for Russian mmlu that many people use to assess model's understanding of the Russian language
@haileyschoelkopf @baberabb @lintangsutawika please, have a look
@lintangsutawika @baberabb hi! Do you mind giving your approve to add mmlu_ru to master?)
Hi! Sorry, missed this. sorry it took so long. If you could add a row to lm_eval/tasks/README.md like all the other tasks, it should be good to merge
Hi! Sorry, missed this. sorry it took so long. If you could add a row to
lm_eval/tasks/README.mdlike all the other tasks, it should be good to merge
Done
@baberabb fixed everything, need your approval and merge