Why is the accuracy of MMMU-COT only 47 when I tested InternVL2_5-38B?

[zzk6626]

Checklist

• [ ] 1. I have searched related issues but cannot get the expected help.
• [ ] 2. The bug has not been fixed in the latest version.
• [ ] 3. Please note that if the bug-related issue you submitted lacks corresponding environment info and a minimal reproducible demo, it will be challenging for us to reproduce and resolve the issue, reducing the likelihood of receiving feedback.

Describe the bug

I tested InternVL2_5-38B on the VLMEvalKit platform using USE_COT="1", but the final accuracy came out to be only around 47%, which is different from the results you reported. I used the default configurations throughout. Could you please let me know if there's anything that needs to be adjusted?

thank you !!

Reproduction

I tested InternVL2_5-38B on the VLMEvalKit platform using USE_COT="1", but the final accuracy came out to be only around 47%, which is different from the results you reported. I used the default configurations throughout. Could you please let me know if there's anything that needs to be adjusted?

thank you !!

Environment

-

Error traceback

[momush]

The same situation also occurs in InterVL2.5_8B.

[zzk6626]

The same situation also occurs in InterVL2.5_8B.

May I ask if you have solved this problem?