intel-extension-for-pytorch icon indicating copy to clipboard operation
intel-extension-for-pytorch copied to clipboard

Under what setting, ipex.llm.optimize will be faster than torch.compile?

Open lostkingdom4 opened this issue 5 months ago • 0 comments

Describe the issue

I was testing on the MAX 1100 and 1550 GPUs using the conda environment set up as described in these instructions. I compared performance between torch.compile and ipex.llm.optimize on float16, and observed that several LLMs perform worse when using torch.compile.

Could you clarify the correct settings or scenarios where ipex.llm.optimize() is expected to outperform torch.compile?

lostkingdom4 avatar Jul 11 '25 17:07 lostkingdom4