Disty0
DDP / Multi GPU is not supported with IPEX. Specify a device in training parameters or try setting one of these environment variable: For IPEX: ``` xpu_VISIBLE_DEVICES=0 ``` For anything...
Are you using Python 3.12? Wipe the venv folder and use 3.10 or 3.11.
What is your GPU or do you have a GPU? xFormers requires an Nvidia GPU. torch.cuda.is_available() is not something we set, it will be True if you have a working...
This is an autocast issue rather than sdpa issue. sdpa is working as intended, autocast should've catched it before it hit sdpa. There are more autocast issues i have worked...
`scaled_dot_product_attention` returns float32 when float16 inputs are used with ipex. bfloat16 inputs returns bfloat16 as expected. This might be related to the ComfyUI issue. Autocast should catch this tho. Diffusers...
Seems to be fixed with IPEX 2.1.20+xpu.
Dig into the NNCF code a bit and found a way to get the original dtype: 2.7.0: https://github.com/openvinotoolkit/nncf/blob/release_v270/nncf/torch/quantization/weights_compression.py#L34 Latest: https://github.com/openvinotoolkit/nncf/blob/9cfc7b48f6511356021801790e725453b2612ac4/nncf/torch/quantization/layers.py#L1059 This returns the original dtype and seems to work fine...
NNCF compression autocasts to original dtype when running the model, so any change we do here doesn't change them. Dtype change in the pipeline is mainly for latents and generator...
We can add something like `self._autocast_dtype` that can be changed from outside to the pipeline but i didn't want to add another variable. Or we can convert the dype variable...
Actually i forgot to check the simplest stuff first. `self.dtype` doesn't get updated by NNCF. This also works. ```py dtype = self.dtype ``` Edit: But breaks again when model_cpu_offload is...