Tri Dao comments

Results 440 comments of


                                            Tri Dao

trafficstars

Function 'FlashAttnQKVPackedFuncBackward' returned nan values in its 0th output

I'm very curious about this. I think all the of values in `dq`, `dk`, `dv` should overwritten during the execution of the backward pass. The only problematic scenario I could...

Function 'FlashAttnQKVPackedFuncBackward' returned nan values in its 0th output

Does the same thing happen if you use standard implementation of attention? i.e. try `use_flash_attn=False`

Does flash attention support FP8?

Not yet for now

Does flash attention support FP8?

The key difficulty is someone needs to implement it :D

Issue with installing flash attention ` import flash_attn_2_cuda as flash_attn_cuda`

You should - Install torch 2.4 (if that's the verison you want) - Install flash-attn (latest version 2.7.0.post2 should work)

[training] `python run.py` raises `ImportError: cannot import name 'GPTBigCodeConfig' from 'transformers'`

Yeah i think the transformers version is the issue.

Why is torch2.9.0 not supported?

can you create a PR?

FlashAttention forward support for Turing

Wow this is great work!

[QST] flash_attn2: why tOrVt is no swizzle ?

Idk tbh. Result was wrong without NoSwizzle.

[QST] flash_attn2: why tOrVt is no swizzle ?

Ha this is great to know, thank you!