ThunderKittens icon indicating copy to clipboard operation
ThunderKittens copied to clipboard

attn_bias rel-pos support to the FAv2 example

Open vadimkantorov opened this issue 1 year ago • 0 comments

AFAIK https://github.com/Dao-AILab/flash-attention/ did not have the bandwidth to support custom attn_bias (needed for relpos) - I think it's supported for the Triton version there, but I saw reports that it's not very stable / performant in backward.

Should it be relatively easy to add in the attention example based on TK?

vadimkantorov avatar May 21 '24 15:05 vadimkantorov