flash-attention icon indicating copy to clipboard operation
flash-attention copied to clipboard

Published 20 hours ago • verified publisher icon Dao-AILab

Relative postitions

Open Sunnikickback opened this issue 1 year ago • 1 comments
trafficstars

Hello, i want to implement flash attention for wavlm, where relative positions are used, i saw an issue, where somebody said it is not supported yet. So question is the same, is it now supported, or i can not to it right now?

Sunnikickback avatar May 11 '24 10:05 Sunnikickback

It is not supported yet.

tridao avatar May 11 '24 19:05 tridao