lighteval icon indicating copy to clipboard operation
lighteval copied to clipboard

[EVAL] Add MathVista

Open ThakurRajAnand opened this issue 5 months ago • 1 comments

Evaluation short description

  • Why is this evaluation interesting? MathVista is one of the most popular and commonly reported benchmark in MultiModal LLM like Qwen2.5_VL and InternVL3_5
  • How used is it in the community? To evaluate multimodal vision reasoning capabilities on Math questions

Evaluation metadata

Provide all available

  • Paper url:https://arxiv.org/pdf/2310.02255
  • Github url: https://github.com/lupantech/MathVista
  • Dataset url: https://huggingface.co/datasets/AI4Math/MathVista

https://mathvista.github.io/

ThakurRajAnand avatar Sep 17 '25 17:09 ThakurRajAnand

hey @ThakurRajAnand @NathanHB can i work on this?

omkar-334 avatar Nov 20 '25 22:11 omkar-334