Unable to reproduce the MMMU effect of think mode

Open sjhdbsmb opened this issue 2 months ago • 1 comments

Excellent work! When I tried to use the Vlmevalkit tool to reproduce the performance of the InternVL3.5 2B model on the MMMU, I found that the evaluation was very slow and there were a lot of repeated messages in the part until the maximum token was reached, and a lot of words such as "Wait, maybe". I would like to know if there are any configuration changes I need to make. The following is my configuration.

Oct 23 '25 03:10 sjhdbsmb

For example, this repetitive phenomenon is very common in , which seriously affects the reasoning speed and the final score is significantly different from the score disclosed in the paper.

Oct 23 '25 09:10 sjhdbsmb