Video-LLaVA icon indicating copy to clipboard operation
Video-LLaVA copied to clipboard

Eval w/ multiple GPUs

Open yaolug opened this issue 1 year ago • 1 comments

why https://github.com/PKU-YuanGroup/Video-LLaVA/blob/main/llava/eval/video/run_inference_video_qa.py#L122 this line is commented out?

yaolug avatar Jan 09 '24 06:01 yaolug

HI @yaolug, I have ran eval with multiple GPUs; notice that when you have multiple GPUs available; https://github.com/PKU-YuanGroup/Video-LLaVA/blob/e93f4927eaa926ed8450b481fde95c994ed23d2d/scripts/v1_5/eval/run_qa_msrvtt.sh#L14-L17

in the forward eval files (for example, in msrvtt): https://github.com/PKU-YuanGroup/Video-LLaVA/blob/main/scripts/v1_5/eval/run_qa_msrvtt.sh

split the dataset into chunks which are combined after the run is finished: https://github.com/PKU-YuanGroup/Video-LLaVA/blob/e93f4927eaa926ed8450b481fde95c994ed23d2d/scripts/v1_5/eval/run_qa_msrvtt.sh#L41-L43

orrzohar avatar Feb 11 '24 00:02 orrzohar