[Bug]: Image generation hangs on the last step of generation. SDXL

[akPetrol]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

When generating images using the SD XL base model, the images start rendering fine, but the last step of the generation takes longer than the rest of the image combined. Tested with regular, medvram and lowvram, multiple samplers and steps.

Testing done with 3060ti, 8 gigs of vram. 32 gigs of ram

Steps to reproduce the problem

1. Go to ....
2. Press ....
3. ...

What should have happened?

Version or Commit where the problem happens

1.5, latest build

What Python version are you running on ?

Python 3.10.x

What platforms do you use to access the UI ?

Windows

What device are you running WebUI on?

No response

Cross attention optimization

xformers

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--autolaunch --xformers --opt-sdp-attention --no-half-vae --opt-channelslast

List of extensions

Extensions disabled

Console logs

Console shows nothing of substance.

Additional information

No response

[dhwz]

Use fp16 fixed VAE, don't use --no-half-vae and make sure you've set --medvram

[bwesen]

I get something similar with a fresh install and sdxl base 1.0, 1024x1024. All iteration steps work fine, and you see a correct preview in the GUI. But when it reaches the end, it gets the dreaded NaN tensor error. After this, auto1111 helpfully tries to re-do using a 32-bit VAE which produces an OOM on my measly 3080 but the underlying problem seems to be the final VAE step to start with.

[dhwz]

@bwesen yeah that's why I recommend using the fp16 VAE

https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/tree/main

[bwesen]

@bwesen yeah that's why I recommend using the fp16 VAE

https://huggingface.co/madebyollin/sdxl-vae-fp16-fix/tree/main

I read the description in the sdxl-vae-fp16-fix README.md, and it seemed to imply that when using the SDXL model loaded on the GPU in fp16 (using .half()), the resulting latents can't be decoded into RGB using the bundled VAE anymore without producing the all-black NaN tensors? And thus you need a special VAE finetuned for the fp16 Unet?

UPD: I tried the linked VAE and it works now, no NaNs! I'm kind of amazed this wasn't fixed previously, I mean auto1111 uses half() when loading the sdxl model by default so everybody should have gotten this error...

I renamed the fpt16 fixed vae as sd_xl_base_1.0.vae.safetensors and put it in the same models folder as the base .safetensors file.

[dhwz]

@bwesen yeah or you put it into the VAE folder so you can switch between different ones (if more get available like on 1.5)

[gaathi1111]

hi... i have the same issue. i did install fp16 VAE, but that didn't solve the issue for me

[swumagic]

ME.TOO

[Kadah]

Try changing Settings > Live Preview > Progressbar/preview update period, in milliseconds to a smaller number, and/or changing the preview mode to a faster method. This sounds a lot like #10110

[swumagic]

txt 2 img Stuck at 95 percent

[Dominux]

@dhwz, oh god, bless ya. I was stuck at the problem of not being able to generate wallpapers for my 3440:1440 monitor with SDXL. Generation itself was great till the end where it failed every goddamn time with the error of lack of VRAM (I got 12GB) I even tried creating my own pipeline with ~~blackjack and hookers~~ the diffusers library from scratch, but I got the same behavior and a consequent result. But once I tried the advice of urs I got the result omg man thx ya I was already thinkin about buying a new card for $2k, ya saved my money for a while bro