AUTOMATIC1111
[Bug]: Image generation crashing: "Tensor with all NaNs was produced in Unet"
Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
What happened?
Image generation stopped at 8-12% and gave the error: "modules.devices.NansException: A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check."
Steps to reproduce the problem
Started a generation I had run previously from PNG Info, so the same generation data was used. Same model as well, Chilloutmix, no VAE. Other models worked fine, but this model worked fine yesterday and I have not changed its version. Have tested multiple other PNG Info's from generations from the same model, same result.
What should have happened?
The image should have generated.
Commit where the problem happens
5ab7f213bec2f816f9c5644becb32eb72c8ffb89
What platforms do you use to access the UI ?
MacOS
What browsers do you use to access the UI ?
Brave
Command Line Arguments
Launching Web UI with arguments: --skip-torch-cuda-test --upcast-sampling --opt-sub-quad-attention --use-cpu interrogate
List of extensions
a1111-sd-webui-lycoris instruct-pix2pix loopback_scaler openpose-editor sd-dynamic-thresholding sd-webui-additional-networks sd-webui-aspect-ratio-helper sd-webui-controlnet sd_webui_SAG ultimate-upscale-for-automatic1111 LDSR | built-in | | Lora | built-in | | ScuNET | built-in | | SwinIR | built-in | | prompt-bracket-checker
Console logs
modules.devices.NansException: A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check.
Additional information
No response
I got this as well just a couple of days ago while doing an upscale in img2img! Sorry I can't help you further, just chiming in so that you know it's not your system!
Also, I'm on Windows so this is not specific to MacOS!
I think this same error has been popping up for other people in different places. Not sure why it just happened for me, I don't have it set to auto update, so my commit should have been the same as before the issue.
I ran into the same problem. Windows 10, RTX 3070 notebook 8GB VRAM. I tried the parameters --no-half, --no-half-vae, --disable-nan-check, turned on the "Upcast cross attention layer to float32" option, but the best result I got was a black image. Then I deleted all parameters, switched to the v-1-5-pruned checkpoint and then the image was generated. From then on, the problem disappeared.
I’ve noticed this since the update to Torch 2 maybe
I’ll get to a point where I get the error every time I press generate, then switching the model to a new one causes the error to stop happening for a bit.
I don’t know the inner workings but maybe some VRAM isn’t getting garbage collected etc?
I know I often get the NaN error when I try go 768x768 but not eg 640x640, so I’m assuming it’s memory related even though I don’t get an OOM error
just guessing though
related: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/10343 https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/10227 https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/9294
A1111 updated for me and now I get this same error everytime I run it. Any updates coming or how can I roll back? modules.devices.NansException: A tensor with all NaNs was produced in Unet. This could be either because there's not enough precision to represent the picture, or because your video card does not support half type. Try setting the "Upcast cross attention layer to float32" option in Settings > Stable Diffusion or using the --no-half commandline argument to fix this. Use --disable-nan-check commandline argument to disable this check.
Duplicate of https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/6923
