Ying Zhang comments

Repositories
Issues
Comments

Results 35 comments of


                                            Ying Zhang

Allow setting a custom TmaDescriptor for TMAStore.

@hwu36 It's fine, I don't need it any more, thx.

[Inductor][ROCm] Composable Kernel backend for Inductor

Thanks @tenpercent , looks good to me overall, could you share benchmark results? How about compilation time?

[Inductor][ROCm] Composable Kernel backend for Inductor

@pytorchbot merge

[FA3] `nan` gradients in the BW pass

Hi @danthe3rd , this problem should have been solved with the new bwd kernel commit. I tried your script with `assert not dq.isnan().any().item()` and didn't observe errors. Could you check...

Support sliding window attention in FA3

Sorry I deleted the original branch by accident. This is the new PR: https://github.com/Dao-AILab/flash-attention/pull/1233.