Simon Lui

> IPEX LLM already supports llama.cpp I think: https://ipex-llm.readthedocs.io/en/latest/doc/LLM/Quickstart/llama_cpp_quickstart.html What IPEX-LLM has is a fork of llama.cpp and some other projects that has optimizations that have not been upstreamed here...

> they seem to be keeping it reasonably up to date, as their published version of LlamaCPP-IPEX is using a week old version as its base line so far. I...

I think this issue was a problem with an older version of IPEX, v2.1.20+xpu, but the latest version should fix things, v2.1.30+xpu, which I haven't ran into yet since upgrading.

Ah okay, apparently, it's still an [open issue](https://github.com/intel/intel-extension-for-pytorch/issues/399), that does somewhat suck for owners until Intel gets up to speed on that. BF16 is faster than FP16 on Intel Arc...

Shoot. I think I had this line wrong where ` xpu_available = xpu_available or (hasattr(torch, "xpu") and torch.xpu.is_available())` should be ` xpu_available = xpu_available and hasattr(torch, "xpu") and torch.xpu.is_available()` Does...

Questions, as an Intel Arc owner and having contributed to the repository. 1.) I have used both the Triton(`inductor`) and OpenVINO backends using custom nodes and Triton is faster in...

Just to note. I have done a custom package of IPEX with `lnl-m` AOT support and the results aren't great. Running the default workflow with Stable diffusion 1.5 in ComfyUI...

@xiguiw Technically, the AOT string was set to `acm-g10,lnl-m` given I have an A770 but yeah, running this on my new Lunar Lake laptop results in issues. If you want,...

@xiguiw The original issue as I presented with the AOT wheels for Linux are still missing for Meteor Lake and above. I am glad there are plans but I don't...

Sorry, I should've seen this earlier but have been busy with personal affairs. I am looking at the logs and it does seem like IPEX does recognize the card and...