iree icon indicating copy to clipboard operation
iree copied to clipboard

[Codegen] Remove --iree-llvmgpu-enable-prefetch

Open Groverkss opened this issue 5 months ago • 0 comments

This flag

https://github.com/iree-org/iree/blob/d834aa7357179e0d806f3634d2efe3af2fa45171/compiler/src/iree/compiler/Codegen/LLVMGPU/KernelConfig.cpp#L90

enables software prefetching for kernels using shared memory. Software prefetching is disabled by default, and only enabled by this flag. Over time, prefetching became part of GPU pipelining options: https://github.com/iree-org/iree/blob/d834aa7357179e0d806f3634d2efe3af2fa45171/compiler/src/iree/compiler/Codegen/Dialect/GPU/IR/IREEGPUAttrs.td#L484 and can be controlled per dispatch.

Possible Solution

We should remove this flag and instead rely on kernel config to decide when to enable this flag. It can usually be enabled for matmul/conv kernels and attention kernels with some analysis based on intrinsic. This flag should ideally be tuned based on profiling, but generally it is okay to enable.

Groverkss avatar Sep 20 '24 19:09 Groverkss