iree
iree copied to clipboard
iree-org
A retargetable MLIR-based machine learning compiler and runtime toolkit.
Closes https://github.com/iree-org/iree/issues/18854
The helper function `isHorizontalToGroup` relies on `getBackwardSlice` which doesn't include operations defined above which are used in the body of an operation (vs as an operand) https://github.com/iree-org/iree/blob/114a1427810f3da0234f98c22f58390773b0489a/compiler/src/iree/compiler/DispatchCreation/FusionUtils.cpp#L104 To be conservative,...
### Tracking issue for llama related `DispatchCreation` changes/improvements #### Reshape related changes: - [ ] Fold `tensor.cast`s with `tensor.expand_shape` ops https://github.com/llvm/llvm-project/pull/112265 - [ ] Bubble `tensor.expand_shape` ops through `tensor.collapse_shape` ops...
Post-softmax, the range of output is between 0, 1. For low-precision types (like fp8), we scale the output range to be between 0, fpMax, so we can use more of...
### What happened? For the given IR ```mlir module { func.func @main_graph(%arg0: !torch.vtensor, %arg1: !torch.vtensor , %arg2:!torch.vtensor , %arg3: !torch.vtensor,%arg5: !torch.vtensor , %arg6: !torch.vtensor) -> !torch.vtensor attributes {torch.onnx_meta.ir_version = 8...
The current tile-selection heuristic in GPUMaterializeEncoding only ever expands to subgroups in the N dimension, never in the M dimension. That allows to keep this logic a little simpler, but...
The tile size selection heuristic in GPUMaterializeEncoding is focused on the generic case of non-narrow shapes; then at the end, a fix-up is applied to adjust to narrow shapes. This...
### What happened? For the attached IR, seeing error as ``` model.modified.mlir:780:12: error: failed to legalize operation 'hal.interface.constant.load' %203 = arith.index_cast %202 : i64 to index ^ model.modified.mlir:14:3: note: called...