LLaVA-NeXT icon indicating copy to clipboard operation
LLaVA-NeXT copied to clipboard

Published 20 hours ago • verified publisher icon LLaVA-VL

Missing mm_projector.bin in LLaVA-Video series

Open kpc0810 opened this issue 1 year ago • 2 comments

Hi, I am trying to fine-tune LLaVA-Video-7B and LLaVA-Video-72B. However, it seems that the checkpoints for mm_projector.bin have not been released yet. Thanks for the help in advance.

kpc0810 avatar Nov 28 '24 12:11 kpc0810

Besides, the training and inference scripts of LLaVA-Video do not load mm_projector. Is my understanding correct? If yes, why did you choose only to load the trained Qwen2?

kpc0810 avatar Nov 28 '24 23:11 kpc0810

i also found this problem.the code looks like only initialize mm_project but not load weight from pretrained model in infrence mode.

huajinghua avatar Apr 22 '25 02:04 huajinghua