returnn
returnn copied to clipboard
Cleanup GaussWindowAttentionLayer
I think this layer is not used by anyone, and obsolete because of the more generic GenericAttentionLayer (or just DotLayer).
Remove?
If you are asking if I ever used it, no I did not use it I think. Even if, my setups all use specific commit hashes. It can be removed in my opinion.
I also tagged you as you might know people or setups which make use of it. Or not. I just wanted to have some confirmation.
I do not know anyone...