intel
[IMPORTANT] The future of Intel® Extension for PyTorch*
Dear Community,
We launched the Intel® Extension for PyTorch* in 2020 with the goal of extending the official PyTorch* to simplify achieving high performance on Intel® CPU and GPU platforms. Over the years, we have successfully upstreamed most of our features and optimizations for Intel® platforms into PyTorch* itself.
Moving forward, our strategy is to focus on developing new features and supporting upcoming platform launches directly within PyTorch*. As a result, we have discontinued active development of the Intel® Extension for PyTorch* and ceased official quarterly releases following the 2.8 release. We strongly recommend using PyTorch* directly going forward, as we remain committed to delivering robust support and performance with PyTorch* for Intel® CPU and GPU platforms.
We will continue to provide critical bug fixes and security patches for two additional quarters to ensure a smooth transition for our partners and the broader community. After that, we plan to mark the project End-of-Life unless there is a solid need to continue maintenance. We will provide further updates as appropriate.
If you have any questions or concerns, please feel free to comment here. Thank you all for your continued support!
If you have any questions
Hello. Does A770 support BFLOAT16 ? (Flex 170 does..)
RuntimeError: Input type (c10::BFloat16) and bias type (c10::Half) should be the same
ValueError: Intel Arc A770 have bfloat16 accuracy known issue. You can use float16 instead by explicitly setting the `dtype` flag in CLI, for example: --dtype=half`
I am trying to start DeepSeek-OCR (https://huggingface.co/deepseek-ai/DeepSeek-OCR) via vllm. pytorch 2.8 (xpu), and ipex 2.8 (https://pytorch-extension.intel.com/?platform=gpu&version=v2.8.10%2Bxpu&os=linux%2Fwsl2&package=pip)
Here is recommendation to use float16: https://github.com/intel/intel-extension-for-pytorch/issues/291#issuecomment-1420237974
A770 generally supports BF16. Typically, we recommend using BF16 for training and FP16 for inference workloads. If you encounter any accuracy issues with specific operators on A770, we suggest testing with the latest PyTorch 2.9 to verify. If the issue persists, please file a report at https://github.com/intel/torch-xpu-ops for further investigation.
Maybe this is not the place to ask. What is the fate of the project https://github.com/intel/ipex-llm? Does Intel want to continue supporting popular inference programs like ollama or llama.cpp?
@danielmayost I don't have the answer. Please ask directly at https://github.com/intel/ipex-llm.
Can a new version of portable Ollama be provided using either latest ipex or Pytorch 2.8 release? Instructions for using the XPU device with Ollama will be helpful.
Can a new version of portable Ollama be provided using either latest ipex or Pytorch 2.8 release? Instructions for using the XPU device with Ollama will be helpful.
I don't have the answer. Please ask directly at https://github.com/intel/ipex-llm.
May I ask how the speed of the pytorch inference model compares with that of ipex-llm now（torch2.8.0+xpu or torch2.9.0+xpu）? Is ipex-llm still faster in the short term? Are there any test results that can be displayed?
@xiaohoua https://github.com/intel/ipex-llm is a separate open source project. For your question, please reach out directly via https://github.com/intel/ipex-llm.
So, Intel is not going to give up continuous support of AI. Instead , the XPU support is merging into the mainstream of Pytorch? So the experience of torch and XPU will be much more smoother?
@ca1ic0 Right, Intel® CPU and GPU hardware support has been upstreamed into native PyTorch*. We remain committed to delivering robust support and performance with PyTorch* for Intel® CPU and GPU platforms.
@ca1ic0 Right, Intel® CPU and GPU hardware support has been upstreamed into native PyTorch*. We remain committed to delivering robust support and performance with PyTorch* for Intel® CPU and GPU platforms.
So, user could directly invoke an issue in Pytorch if they have problem in precision and performance when using torch with intel accelerators. Thanks for your works!