lm-evaluation-harness icon indicating copy to clipboard operation
lm-evaluation-harness copied to clipboard

Added TurkishMMLU to LM Evaluation Harness

Open ArdaYueksel opened this issue 1 year ago • 1 comments

In this pull request, I would like to add our work TurkishMMLU: Measuring Massive Multitask Language Understanding in Turkish to LM Evaluation Harness. You can find the details of our work in our repository: https://github.com/ArdaYueksel/TurkishMMLU Also, our dataset is made available in HuggingFace: https://huggingface.co/datasets/AYueksel/TurkishMMLU

Key Features:

  • MMLU variant in Turkish Language
  • Has a separate Development Set.
  • Chain-of-Thought Configuration is available.

ArdaYueksel avatar Sep 06 '24 13:09 ArdaYueksel

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Sep 06 '24 13:09 CLAassistant

Test failures unrelated.

haileyschoelkopf avatar Sep 26 '24 19:09 haileyschoelkopf