Ask-Anything sequential ` for samples in batch` loop in Videochat2 is slow, any ways to parallalize it ?

sequential ` for samples in batch` loop in Videochat2 is slow, any ways to parallalize it ?

Open dragen1860 opened this issue 3 months ago • 1 comments

Hi, dear all: I noticed a sequential for loop in videochat2_it.py forward function:

https://github.com/OpenGVLab/Ask-Anything/blob/078540aaebfbe1ad9a109020a73b0ce173b355ef/video_chat2/models/videochat2_it.py#L240-L288

As most of tensor computation is parallel, any good idea to implement the for loop parallel?

Mar 12 '24 07:03 dragen1860