Orr Zohar

Yeah, I made the same edit on my local repo. When do you use the _act eval inference file? Also, I am getting a module import error (videollava) trying to...

I mean, an easy variation would be to use Vicuna as the weights are open-source it would be more comparable... At the very least, it would make sense to set...

By the way; when I evaluate TGIF: Yes count: 9249 No count: 16502 Accuracy: 0.3591705176498 Average score: 2.5519785639392647

I didn't change your eval files; the default you use is GPT3.5: https://github.com/PKU-YuanGroup/Video-LLaVA/blob/e93f4927eaa926ed8450b481fde95c994ed23d2d/videollava/eval/video/eval_video_qa.py#L39

The reason I think you may have uploaded the wrong model to transformers is that I get the following (top row is the model you released; bottom row is a...

HI @yaolug, I have ran eval with multiple GPUs; notice that when you have multiple GPUs available; https://github.com/PKU-YuanGroup/Video-LLaVA/blob/e93f4927eaa926ed8450b481fde95c994ed23d2d/scripts/v1_5/eval/run_qa_msrvtt.sh#L14-L17 in the forward eval files (for example, in msrvtt): https://github.com/PKU-YuanGroup/Video-LLaVA/blob/main/scripts/v1_5/eval/run_qa_msrvtt.sh split the...