Horace He
Results
242
comments of
Horace He
@casper-hansen On H100, F.scaled_dot_product_attention calls into CuDNN attention, which has a much smaller gap in performance with FA3.
I performed the tests listed at "Typing with pleasure" on my own computer, and these are my results: 