Kwan Kin Chan comments

Repositories
Issues
Comments

Results 3 comments of


                                            Kwan Kin Chan

Issues with Running video_chat2 on Multi-GPU Setup with Nvidia Titan Xp

Hi, I tried to load the model with dual 4090 and still faced the same error after applying the changes. I looked into debugger and realized that it is because...

Fail to run v2 with flash attention

I added the `use_cache = False` at https://github.com/OpenGVLab/Ask-Anything/blob/078540aaebfbe1ad9a109020a73b0ce173b355ef/video_chat2/conversation.py#L64-L75 and I get a new error message. ``` Exception has occurred: RuntimeError shape '[-1, 125]' is invalid for input of size 126...

Fail to run v2 with flash attention

yes, I was able to run the model without `flash_attn`. However, I am trying flash attention because I want a faster and more memory-efficient inference when using long prompts. Apart...