dlrover icon indicating copy to clipboard operation
dlrover copied to clipboard

Published 20 hours ago • verified publisher icon intelligent-machine-learning

How to use Flash Attention x TFPlus?

Open fuhailin opened this issue 1 year ago • 1 comments

Flash Attention x TFPlus is a nice work, and how to use it? Could you please provide an example or unit test about Flash Attention?

fuhailin avatar Nov 15 '23 05:11 fuhailin

You can refer to this commit to use MultiHeadAttention like in TensorFlow. https://github.com/intelligent-machine-learning/dlrover/pull/850

mars1248 avatar Nov 20 '23 11:11 mars1248