VLMEvalKit icon indicating copy to clipboard operation
VLMEvalKit copied to clipboard

[Request]Consider integrating MMT-Bench and CONTEXTUAL?

Open iamlockelightning opened this issue 9 months ago • 1 comments

MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI https://arxiv.org/pdf/2404.16006

CONTEXTUAL: Evaluating Context-Sensitive Text-Rich Visual Reasoning in Large Multimodal Models https://arxiv.org/pdf/2401.13311

iamlockelightning avatar May 10 '24 02:05 iamlockelightning

Thanks for your interest in MMT-Bench. We will integrate MMT-Bench ASAP.

KainingYing avatar May 10 '24 03:05 KainingYing

Hi, @iamlockelightning MMT-Bench has already been supported.

kennymckormick avatar Jun 18 '24 07:06 kennymckormick