VLMEvalKit icon indicating copy to clipboard operation
VLMEvalKit copied to clipboard

Request to add SIBench evaluation code

Open song2yu opened this issue 4 months ago • 1 comments

Added evaluation code for the SIBench paper: "How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspective https://arxiv.org/abs/2509.18905 ".

  1. Includes inference_mixed.py to support mixed inference for both images and videos.

  2. Includes SIBench.py for processing the SIBenchmark.

  3. Introduced a new MixedOutput format.

  4. Added post-processing support for the MixedOutput format in run.py.

song2yu avatar Nov 09 '25 15:11 song2yu

Please fix the lint issue

tonysy avatar Nov 12 '25 09:11 tonysy

Thanks for the reminder. I have already checked the code according to the development guide pre-commit run --all-files, and now shows no formatting errors.

song2yu avatar Nov 18 '25 07:11 song2yu