[bug]: Converted diffuser model do not get the same/close image with same prompt and seed

[i3oc9i]

Is there an existing issue for this?

• [X] I have searched the existing issues

OS

macOS

GPU

mps

VRAM

128G

What happened?

I'have tried to convert a safetensors model to a diffuser one, then I replay exactly the same prompt, with the same settings, and same seed.

The result image is very far from the safetensors one, I am expecting the same or very close result.

IMHO, the issue is that converting the model, lose same model properties...

Screenshots

The following images were generated with https://civitai.com/api/download/models/4694 (QGO-MK8)

original safetensors

diffuser one

Additional context

No response

Contact Details

No response

[i3oc9i]

FYI, the VAE related to the Kalista.ckpt model is the usual vae-ft-mse-840000-ema-pruned.ckpt)

If I answer No to the VAE question during the optimisation process, then I get a desaturated/fade image corresponding to the one of diffuser model reported in the issue.

[lstein]

The issue appears to be related to xformers, which is enabled for diffusers models and disabled for legacy checkpoints. If you launch with --no-xformers then the images from the converted and original models are almost the same. They're not identical, but pretty close.

Here is "banana sushi" using the civitai mix: pastelMixStylizedAnime_pastelMixPrunedFP16

original version:

Diffusers version - xcode off

Diffusers version - xcode on

[i3oc9i]

good point for PC/Linux users, but I'm on MacOs so xformers are not installed.. :) ( I have anyway started the invoke how you suggested with the --no-xformers)

I was getting crazy about this issue, but I think I get the point. I did more test with the kalista (from civitai) model.

banana sushi -s 29 -S 123456789 -W 512 -H 512 -C 7.5 -A k_lms banana sushi -s 30 -S 123456789 -W 512 -H 512 -C 7.5 -A k_lms [OK]

banana sushi -s 29 -S 123456789 -W 512 -H 512 -C 7.5 -A k_euler_a banana sushi -s 30 -S 123456789 -W 512 -H 512 -C 7.5 -A k_euler_a [OK]

banana sushi -s 29 -S 123456789 -W 512 -H 512 -C 7.5 -A k_heun banana sushi -s 30 -S 123456789 -W 512 -H 512 -C 7.5 -A k_heun [OK]

So Keras diffusers are different under 30 steps..., but they match at 30 steps and over

ddim and plms diffusers are still different at 100 steps

[github-actions[bot]]

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

[github-actions[bot]]

Due to inactivity, this issue was automatically closed. If you are still experiencing the issue, please recreate the issue.