VLMEvalKit icon indicating copy to clipboard operation
VLMEvalKit copied to clipboard

fix(image_mcp.py): use consistant argument (#929)

Open MaoSong2022 opened this issue 8 months ago • 2 comments

fix argument error in MMMU_Pro benchmark, close #929

MaoSong2022 avatar Apr 28 '25 12:04 MaoSong2022

Hi, @MaoSong2022 ,

It looks like every other dataset do not set the arg names explicitly, so I don't think the modification is necessary.

kennymckormick avatar Apr 28 '25 14:04 kennymckormick

the change is to keep the arguments consistent with ImageBaseDataset initialization. For now, only a few dataset implements this method.

  • ImageBaseDataset initialization: def __init__(self, dataset='MMBench', skip_noimg=True)
  • some other dataset initialization (such as MMGenBench): def __init__(self, dataset, **kwargs)
  • this commit (related issue): def __init__(self, dataset='MMMU_Pro', skip_noimg=True)

the question is should we use consistent arguments when inheriting ImageBaseDataset, if the answer is yes, which format should we use.

MaoSong2022 avatar Apr 29 '25 03:04 MaoSong2022