Zizheng Pan comments

Results 7 comments of


                                            Zizheng Pan

Codes about using performer as the attetion module.

Any updates on this issue? I'd also like to know how you apply Performer into your architecture, as shown in Table 5&6.

Retrained model does not get the same SPL on val unseen as reported in paper

Thanks for your reply!

Retrained model does not get the same SPL on val unseen as reported in paper

Hi @jingquanliang , I didn't see your result, have you fixed it? Or you can try this script. name=agent_bt flag="--attn soft --train validlistener --load snap/agent_bt/state_dict/best_val_unseen --angleFeatSize 128 --submit --featdropout 0.4...

InternVL2-8B and InternVL2-26B CUDA error: an illegal memory access was encountered

Thanks for the help @czczup, I just tried this new code and unfortunately it still doesn't work. My setting is based on 8 GPUs and I can run other scales...

Question about training speed

Hi @tanbuzheng, thanks for using HiLo. Can you give more details or a brief script of your testing? e.g., tensor shape of your feature map, concrete setting of HiLo including...

Question about training speed

Thanks for the addtional information. If we only look at the Attention layer itself, theoretically HiLo attention should be faster than Local Window Attention, ```python from hilo import HiLo W...

How to use videos as input for HiLo？

Hi @q958287831, thank you for your interest! A video clip can be thought of as a list of frames. Initially, for HiLo, we have an input tensor in the shape...