lm-evaluation-harness icon indicating copy to clipboard operation
lm-evaluation-harness copied to clipboard

add Russian mmlu

Open tatiana-iazykova opened this issue 1 year ago • 7 comments

tatiana-iazykova avatar Oct 03 '24 07:10 tatiana-iazykova

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

:white_check_mark: tatiana-iazykova
:x: Tatyana Yazykova


Tatyana Yazykova seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

CLAassistant avatar Oct 03 '24 07:10 CLAassistant

@haileyschoelkopf @lintangsutawika @baberabb do you mind to have a look at it? I just took existing mmlu code and adapted it for Russian mmlu that many people use to assess model's understanding of the Russian language

tatiana-iazykova avatar Oct 03 '24 08:10 tatiana-iazykova

@haileyschoelkopf @baberabb @lintangsutawika please, have a look

tatiana-iazykova avatar Nov 01 '24 09:11 tatiana-iazykova

@lintangsutawika @baberabb hi! Do you mind giving your approve to add mmlu_ru to master?)

tatiana-iazykova avatar Nov 05 '24 08:11 tatiana-iazykova

Hi! Sorry, missed this. sorry it took so long. If you could add a row to lm_eval/tasks/README.md like all the other tasks, it should be good to merge

baberabb avatar Dec 04 '24 16:12 baberabb

Hi! Sorry, missed this. sorry it took so long. If you could add a row to lm_eval/tasks/README.md like all the other tasks, it should be good to merge

Done

tatiana-iazykova avatar Dec 04 '24 16:12 tatiana-iazykova

@baberabb fixed everything, need your approval and merge

tatiana-iazykova avatar Dec 04 '24 20:12 tatiana-iazykova