LLaVA-NeXT icon indicating copy to clipboard operation
LLaVA-NeXT copied to clipboard

Code Bug for llava/model/llava_arch.py

Open Jerrrrykun opened this issue 1 year ago • 3 comments

https://github.com/LLaVA-VL/LLaVA-NeXT/blob/b3a46be22d5aa818fa1a23542ae3a28f8e2ed421/llava/model/llava_arch.py#L230

Not every model config has the attribute "add_faster_video" (e.g. https://huggingface.co/lmms-lab/llava-onevision-qwen2-0.5b-ov/blob/main/config.json). People have to add an overwrite config parameter to their codes manually. I do not see this as intended (?)

Maybe should add hasattr() for further judgment.

Jerrrrykun avatar Oct 03 '24 03:10 Jerrrrykun

https://github.com/LLaVA-VL/LLaVA-NeXT/blob/b3a46be22d5aa818fa1a23542ae3a28f8e2ed421/llava/model/llava_arch.py#L230

Not every model config has the attribute "add_faster_video" (e.g. https://huggingface.co/lmms-lab/llava-onevision-qwen2-0.5b-ov/blob/main/config.json). People have to add an overwrite config parameter to their codes manually. I do not see this as intended (?)

Maybe should add hasattr() for further judgment.

Agreed. But even in the Video-7B-Qwen2, the add_faster_video is set to False, which makes me wonder whether the slow-fast implementation is already supported?

countytown avatar Oct 05 '24 11:10 countytown

I have the same question, has anyone found the answer?

zhipeixu avatar Mar 11 '25 04:03 zhipeixu

It looks like they have commented this out in the code, so I am also wondering whether the released models have actually used the slow-fast features as described in the paper! (https://github.com/LLaVA-VL/LLaVA-NeXT/blob/09e5840d5589ad2d6a8656c0a60f21ae134b3309/llava/model/llava_arch.py#L291C32-L291C55)

sam-motamed avatar Apr 18 '25 13:04 sam-motamed