LLaVA-NeXT icon indicating copy to clipboard operation
LLaVA-NeXT copied to clipboard

Published 20 hours ago • verified publisher icon LLaVA-VL

Failure to reproduce the paper results

Open yuan-QAQ opened this issue 1 year ago • 1 comments

I cloned the "lmms-lab/LLaVA-NeXT-Interleave-Bench" dataset and "llava-onevision-qwen2-7b-ov" checkpoint from Huggingface to reproduce the results of the paper, but some benchmark results seem to be very different (e.g. IEI, qbench, 3D-Chat, MathVerse, SciVerse). What could be the reason for this?

image

yuan-QAQ avatar Aug 22 '24 03:08 yuan-QAQ

May I know what's your reproduce pipeline and specific script?

Luodian avatar Aug 25 '24 12:08 Luodian