tilelang
tilelang copied to clipboard
[Question] Is flash attention supported in sm100 with head dimension 256?
Required prerequisites
- [x] I have read the documentation https://tilelang.com.
- [x] I have searched the Issue Tracker that this hasn't already been reported. (comment there if it has.)
Questions
Hi, I plan to run our model on B200 (don't have it yet) and would like to confirm whether tile-lang Flash Attention can handle head dim 256 on sm100 before we get access to the B200 nodes. A quick confirmation would save us a lot trouble/cost. Much appreciate