Support L2 cache hint

[manman-ren]

This patch currently creates a cache policy for each load/store with eviction policy. It may be better to create cache policy globally for each function.

Fixes https://github.com/openai/triton/issues/3438

[manman-ren]

Sorry, I have forgotten about the refactoring. Will convert to draft and do an internal review first.

[htyu]

Thanks for working on this. LGTM.

I'm also curious about how this moves perf. Maybe kick off a Pytorch nightly perf run?

[manman-ren]

What is the easiest way to kick off pytorch perf test? Is it changing the pytorch pin to this hash, then start some job from the diff? @htyu

[htyu]

What is the easiest way to kick off pytorch perf test? Is it changing the pytorch pin to this hash, then start some job from the diff? @htyu

Yes. Once you update pytorch pin with this one, push it to a personal branch in pytorch repro and submit a perf run on a branch by using:

https://github.com/pytorch/pytorch/actions/workflows/inductor-perf-test-nightly.yml

Then see the result ~12 hours later in: https://hud.pytorch.org/benchmark/compilers

[alexanderwh]

how is the perf result?

[manman-ren]

how is the perf result?

Sorry, still didn't get a chance to measure it on pytorch.