triton icon indicating copy to clipboard operation
triton copied to clipboard

[Backend] Improve dot support to target FMA

Open binarman opened this issue 6 months ago • 1 comments

This PR:

  • Refactors FMA dot implementation
  • Supports dot3d in FMA path
  • Fixes several issues in operand offset computation
  • Enables small dot operands

This PR is a part of PR series. Final goal is to improve efficiency of small dot operations and bypass as much shared memory accesses as possible.

Rough list of PRs:

  • [ ] Basic FMA dot fixes, dot 3d support and relaxing small dimensions for dot (this PR) #4516
  • [ ] Blocked->dotOp shared memory bypassing #4538
  • [ ] Accelerate AMD Matmul + emit dot operations #4594
  • [ ] Layout optimization, so operand B is loaded in proper mfma layout and do not need to go through LDS #4581
  • [ ] Vectorization optimization of dot operands/results (in case llvm can not do this internally)
  • [ ] Reduction operation hoisting out of the K loop (reduction operation is a byproduct of layout optimization step) #4559

binarman avatar Aug 14 '24 13:08 binarman