iree
iree copied to clipboard
[Codegen] Remove --iree-llvmgpu-enable-prefetch
This flag
https://github.com/iree-org/iree/blob/d834aa7357179e0d806f3634d2efe3af2fa45171/compiler/src/iree/compiler/Codegen/LLVMGPU/KernelConfig.cpp#L90
enables software prefetching for kernels using shared memory. Software prefetching is disabled by default, and only enabled by this flag. Over time, prefetching became part of GPU pipelining options: https://github.com/iree-org/iree/blob/d834aa7357179e0d806f3634d2efe3af2fa45171/compiler/src/iree/compiler/Codegen/Dialect/GPU/IR/IREEGPUAttrs.td#L484 and can be controlled per dispatch.
Possible Solution
We should remove this flag and instead rely on kernel config to decide when to enable this flag. It can usually be enabled for matmul/conv kernels and attention kernels with some analysis based on intrinsic. This flag should ideally be tuned based on profiling, but generally it is okay to enable.