triton icon indicating copy to clipboard operation
triton copied to clipboard

Layout conversion bypass for blocked to dotOperand

Open binarman opened this issue 6 months ago • 0 comments

This PR extends shared memory bypass for blocked->dot operand conversions and adds bypass check in DecomposeUnsupportedConversions and ReduceDataDuplication.

This PR is a part of PR series. Final goal is to improve efficiency of small dot operations and bypass as much shared memory accesses as possible.

Rough list of PRs:

  • [ ] Basic FMA dot fixes, dot 3d support and relaxing small dimensions for dot #4516
  • [ ] Blocked->dotOp shared memory bypassing (this PR)#4538
  • [ ] Accelerate AMD Matmul + emit dot operations #4594
  • [ ] Layout optimization, so operand B is loaded in proper mfma layout and do not need to go through LDS #4581
  • [ ] Vectorization optimization of dot operands/results (in case llvm can not do this internally)
  • [ ] Reduction operation hoisting out of the K loop (reduction operation is a byproduct of layout optimization step) #4559

binarman avatar Aug 19 '24 14:08 binarman