intel-extension-for-pytorch
intel-extension-for-pytorch copied to clipboard
intel
AOT IPEX wheels for Meteor Lake and above targeting Linux missing from releases.
Describe the issue
I have a laptop with Lunar Lake coming in and I was interested in having Linux run on it with IPEX. However, going to the release URL for IPEX specific for Lunar Lake, there are no wheels for Linux at all, only Windows versions. And in fact, going to the other URLs available reveals that any wheels for Meteor Lake and above is missing for Linux, only the main xpu release has Linux wheels. And inspecting the latest IPEX release from there, it seems like the current wheel only supports Alchemist given the string used for USE_AOT_DEVLIST.
❯ python
Python 3.11.10 | packaged by conda-forge | (main, Oct 16 2024, 01:27:36) [GCC 13.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
^R
>>> import intel_extension_for_pytorch
[W1219 18:50:26.199322812 OperatorEntry.cpp:155] Warning: Warning only once for all operators, other operators may also be overridden.
Overriding a previously registered kernel for the same operator and the same dispatch key
operator: aten::_cummax_helper(Tensor self, Tensor(a!) values, Tensor(b!) indices, int dim) -> ()
registered at /build/pytorch/build/aten/src/ATen/RegisterSchema.cpp:6
dispatch key: XPU
previous kernel: registered at /build/pytorch/build/aten/src/ATen/RegisterCPU.cpp:30476
new kernel: registered at /build/intel-pytorch-extension/build/Release/csrc/gpu/csrc/aten/generated/ATen/RegisterXPU.cpp:2971 (function operator())
>>> getattr(intel_extension_for_pytorch, '__build_aot__')
'ats-m150,pvc'
I can compile so no issue for me to create a wheel of my own but I was wondering if it is planned to provide any support on Linux for IPEX on Meteor Lake and higher at least for AOT specific builds. Thanks.
Hi @simonlui , thanks for using Intel GPU and thanks for the feedback, we are considering this request for next releases.
Just to note. I have done a custom package of IPEX with lnl-m AOT support and the results aren't great. Running the default workflow with Stable diffusion 1.5 in ComfyUI yields a runtime error and never finishes the diffusion section at 85% nor gets to the VAE Decode section.
got prompt
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
Requested to load SD1ClipModel
loaded completely 9.5367431640625e+25 235.84423828125 True
2024-12-20 14:01:14,339 - _logger.py - IPEX - INFO - Currently split master weight for xpu only support sgd
2024-12-20 14:01:14,346 - _logger.py - IPEX - INFO - Conv BatchNorm folding failed during the optimize process.
2024-12-20 14:01:14,349 - _logger.py - IPEX - INFO - Linear BatchNorm folding failed during the optimize process.
2024-12-20 14:01:14,349 - _logger.py - IPEX - WARNING - [NotSupported]failed to apply concat_linear on unet, please report bugs
2024-12-20 14:01:14,818 - _logger.py - IPEX - INFO - Currently split master weight for xpu only support sgd
2024-12-20 14:01:14,822 - _logger.py - IPEX - INFO - Conv BatchNorm folding failed during the optimize process.
2024-12-20 14:01:14,825 - _logger.py - IPEX - INFO - Linear BatchNorm folding failed during the optimize process.
2024-12-20 14:01:14,826 - _logger.py - IPEX - WARNING - [NotSupported]failed to apply concat_linear on unet, please report bugs
Requested to load BaseModel
loaded completely 9.5367431640625e+25 1639.406135559082 True
2024-12-20 14:01:15,990 - _logger.py - IPEX - INFO - Currently split master weight for xpu only support sgd
2024-12-20 14:01:15,996 - _logger.py - IPEX - INFO - Conv BatchNorm folding failed during the optimize process.
2024-12-20 14:01:16,003 - _logger.py - IPEX - INFO - Linear BatchNorm folding failed during the optimize process.
2024-12-20 14:01:16,004 - _logger.py - IPEX - WARNING - [NotSupported]failed to apply concat_linear on unet, please report bugs
85%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▏ | 17/20 [00:13<00:02, 1.29it/s]
!!! Exception during processing !!! Native API failed. Native API returns: 2147483646 (UR_RESULT_ERROR_UNKNOWN)
Traceback (most recent call last):
File "/home/simonlui/Code_Repositories/ComfyUI/execution.py", line 328, in execute
output_data, output_ui, has_subgraph = get_output_data(obj, input_data_all, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/execution.py", line 203, in get_output_data
return_values = _map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True, execution_block_cb=execution_block_cb, pre_execute_cb=pre_execute_cb)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/execution.py", line 174, in _map_node_over_list
process_inputs(input_dict, i)
File "/home/simonlui/Code_Repositories/ComfyUI/execution.py", line 163, in process_inputs
results.append(getattr(obj, func)(**inputs))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/nodes.py", line 1502, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/nodes.py", line 1469, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 22, in informative_sample
raise e
File "/home/simonlui/Code_Repositories/ComfyUI/custom_nodes/ComfyUI-Impact-Pack/modules/impact/sample_error_enhancer.py", line 9, in informative_sample
return original_sample(*args, **kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/sample.py", line 43, in sample
samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 1013, in sample
return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 911, in sample
return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 897, in sample
output = executor.execute(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/patcher_extension.py", line 110, in execute
return self.original(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 866, in outer_sample
output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 850, in inner_sample
samples = executor.execute(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/patcher_extension.py", line 110, in execute
return self.original(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 707, in sample
samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/k_diffusion/sampling.py", line 155, in sample_euler
denoised = model(x, sigma_hat * s_in, **extra_args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 379, in __call__
out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 832, in __call__
return self.predict_noise(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 835, in predict_noise
return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 359, in sampling_function
out = calc_cond_batch(model, conds, x, timestep, model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 195, in calc_cond_batch
return executor.execute(model, conds, x_in, timestep, model_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/patcher_extension.py", line 110, in execute
return self.original(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/samplers.py", line 308, in _calc_cond_batch
output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/model_base.py", line 129, in apply_model
return comfy.patcher_extension.WrapperExecutor.new_class_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/patcher_extension.py", line 110, in execute
return self.original(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/model_base.py", line 158, in _apply_model
model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 831, in forward
return comfy.patcher_extension.WrapperExecutor.new_class_executor(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/patcher_extension.py", line 110, in execute
return self.original(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 908, in _forward
h = forward_timestep_embed(module, h, emb, context, transformer_options, output_shape, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/ldm/modules/diffusionmodules/openaimodel.py", line 44, in forward_timestep_embed
x = layer(x, context, transformer_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/ldm/modules/attention.py", line 767, in forward
x = block(x, context=context[i], transformer_options=transformer_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/ldm/modules/attention.py", line 654, in forward
n = self.attn1(n, context=context_attn1, value=value_attn1)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/ldm/modules/attention.py", line 540, in forward
k = self.to_k(context)
^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/ops.py", line 70, in forward
return super().forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/torch/nn/modules/linear.py", line 125, in forward
return F.linear(input, self.weight, self.bias)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Native API failed. Native API returns: 2147483646 (UR_RESULT_ERROR_UNKNOWN)
Prompt executed in 17.36 seconds
Exception in thread Thread-4 (prompt_worker):
Traceback (most recent call last):
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/threading.py", line 1075, in _bootstrap_inner
self.run()
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/threading.py", line 1012, in run
self._target(*self._args, **self._kwargs)
File "/home/simonlui/Code_Repositories/ComfyUI/main.py", line 204, in prompt_worker
comfy.model_management.soft_empty_cache()
File "/home/simonlui/Code_Repositories/ComfyUI/comfy/model_management.py", line 1073, in soft_empty_cache
torch.xpu.empty_cache()
File "/home/simonlui/.conda/envs/comfyui/lib/python3.12/site-packages/intel_extension_for_pytorch/xpu/memory.py", line 22, in empty_cache
intel_extension_for_pytorch._C._emptyCache()
RuntimeError: Native API failed. Native API returns: 20 (UR_RESULT_ERROR_DEVICE_LOST)
Either I don't have the lnl-m support that was used to build the Windows wheel or more development time is needed before it is ready at the time of writing this. Either way, I guess that explains why at least Lunar Lake isn't there yet. I am surprised it is being released at all in a beta state on Windows.
Just to note. I have done a custom package of IPEX with
lnl-mAOT support and the results aren't great. Running the default workflow with Stable diffusion 1.5 in ComfyUI yields a runtime error and never finishes the diffusion section at 85% nor gets to the VAE Decode section.
@simonlui
Do you mean you build the package of IPEX with AOT set lnl-m?
Did you run and get the following error in Linux?
@xiguiw Technically, the AOT string was set to acm-g10,lnl-m given I have an A770 but yeah, running this on my new Lunar Lake laptop results in issues. If you want, I can open a separate issue for this particular issue since this is barely related to the issue title.
@simonlui
No, this is expected behavior. We don't need to track this issue.
Could we close this ticket?
@xiguiw The original issue as I presented with the AOT wheels for Linux are still missing for Meteor Lake and above. I am glad there are plans but I don't want to close it until they are there.
@simonlui OK. That's fine.
I will update the released message you to notify when the AOT wheel for Linux released for Meteor Lake.