MA-LMM icon indicating copy to clipboard operation
MA-LMM copied to clipboard

Published 20 hours ago • verified publisher icon boheumd

Run long video rightly

Open shuyansy opened this issue 9 months ago • 5 comments

Hi, thanks for your great work! Now I am running the demo and want to expand the frames (larger than 20). what parameters should I just？the one in the load_video function or the one in the yaml? Thanks for your time!

shuyansy avatar Apr 30 '24 08:04 shuyansy

Also, I want to know how to adjust the "memory_bank_length: 10 num_frames: 20 " When I expand them, the errors will occur. 截屏2024-04-30 下午5 27 40

shuyansy avatar Apr 30 '24 09:04 shuyansy

Hello, I have updated the demo.ipynb in the latest version. You can easily specify the memory_bank_length and num_frames when loading the model. Please note that, every time you change the memory_bank_length or num_frames, you need to reload the model again.

boheumd avatar Apr 30 '24 16:04 boheumd

Hi, thanks for the update.However, when I want to load more frames in model , this errors occurs. Do you have any ideas?

截屏2024-05-01 下午12 38 27 截屏2024-05-01 下午12 38 35

shuyansy avatar May 01 '24 04:05 shuyansy

Hello, please check for the latest code update. Currently the max_num_frames is set to 120 by default. If you need to test model on long videos, you need to set the max_num_frames to a larger value in lavis/configs/models/blip2/blip2_instruct_vicuna7b.yaml or lavis/configs/models/blip2/blip2_instruct_vicuna13b.yaml

boheumd avatar May 02 '24 02:05 boheumd

It works well now for me. Thanks again for the great work and your effort!

shuyansy avatar May 02 '24 06:05 shuyansy