VLMEvalKit icon indicating copy to clipboard operation
VLMEvalKit copied to clipboard

difference sample number with HallusionBench's paper

Open nutsintheshell opened this issue 7 months ago • 0 comments

I try to evaluate models on HallusionBench. The number of hallusionbench in lmevalkit is 951, which is 1129 in HallusionBench's huggingface. I wonder why they have different number. What's more, I get only about 30 score of seed1.5vl on hallusionbench, which is far from 60.3 in the seed1.5vl paper. Could someone get the similar result with me? Thanks.

nutsintheshell avatar May 31 '25 12:05 nutsintheshell