Open-Sora
Open-Sora copied to clipboard
hpcaitech
caption_llava.py output empty information
When using liuhaotian/llava-v1.6-vicuna-7b or liuhaotian/llava-v1.6-mistral-7b for video caption, there is a very high chance to output empty information. Changing temperature from 0.2 to 0.8 helps with the problem, but still, the model cannot deal with all the videos.
In our experiments, we only use the 34b model as the 7b model's ability is poor. Thus, we do not test the 7b's quality. I am concerned that the 7b's quality cannot satisfied video generation requirement.
me too, output.csv empty information. use the llava-v1.6-vicuna-7b,
The input token length exceeds the maximum token length of the 7b model.
me too, output.csv empty information. use the llava-v1.6-vicuna-7b,
The input token length exceeds the maximum token length of the 7b model.
How can I fix this problem?
