visualization the attention map

Open yancx8 opened this issue 1 year ago • 2 comments

Thank you for your work! I want visualization the result of cross attention in BasicTransformerBlock, how can i do? I tried get the attention map, but there is no parameter "need_weights" . Looking forward for your answer!

Nov 13 '24 03:11 yancx8

You may check xformers to acquire the attention map. xformers adopts a block-wise calculation where no explicit "map" is stored in the memory.

Nov 18 '24 08:11 flymin

This issue is stale because it has been open for 7 days with no activity. If you do not have any follow-ups, the issue will be closed soon.

Nov 25 '24 16:11 github-actions[bot]