[Bug]: OutOfMemoryError: CUDA out of memory

[StepHaze]

Checklist

• [ ] The issue exists after disabling all extensions
• [X] The issue exists on a clean installation of webui
• [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
• [ ] The issue exists in the current version of the webui
• [ ] The issue has not been reported before recently
• [ ] The issue has been reported before but has not been fixed yet

What happened?

txt2img doesn't work as expected.

I installed everything, the browser opened 127.0.0.1:7860, I typed a simple prompt, selected the only checkpoint (v1-5-pruned-emaonly.safetensors), clicked Generate button and got:

OutOfMemoryError: CUDA out of memory. Tried to allocate 58.00 MiB (GPU 0; 3.81 GiB total capacity; 2.82 GiB already allocated; 13.44 MiB free; 2.90 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Time taken: 9.1 sec. A: 2.82 GB, R: 2.90 GB, Sys: 3.8/3.8125 GB (99.7%)

Please give me tips and tricks to avoid this error and generate images. Which options should I change at first? Thank you.

Steps to reproduce the problem

1. On the tab txt2img type a simple prompt
2. Click Generate button
3. Get the error.

What should have happened?

I need to generate AI-images.

What browsers do you use to access the UI ?

Mozilla Firefox

[thiagojramos]

Try adding the "PYTORCH_CUDA_ALLOC_CONF" variable to the "webui-user.bat" file.

Example

@echo off
set PYTHON="C:\MiniConda3\envs\WebUI\python.exe"
set GIT=
set VENV_DIR=-
set COMMANDLINE_ARGS= --api --xformers --update-all-extensions  --disable-safe-unpickle --enable-insecure-extension-access --no-download-sd-model --skip-install --precision full --no-half
set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.9,max_split_size_mb:512
set SAFETENSORS_FAST_GPU=1
::set SADTALKER_CHECKPOINTS="C:\AI\Models\SadTalker\checkpoints"
set TF_ENABLE_ONEDNN_OPTS=0
set DREAMBOOTH_SKIP_INSTALL=True
set SD_WEBUI_LOG_LEVEL=INFO
set ACCELERATE=True
set NUMEXPR_MAX_THREADS=9
::set REQS_FILE=.\\extensions\\sd_dreambooth_extension\\requirements.txt
git pull
call webui.bat

[StepHaze]

Try adding the "PYTORCH_CUDA_ALLOC_CONF" variable to the "webui-user.bat" file.

Example

Thank you for your help. Now my webui-user.bat file looks so: @echo off set PYTHON= set GIT= set VENV_DIR= set COMMANDLINE_ARGS= set PYTORCH_CUDA_ALLOC_CONF=garbage_collection_threshold:0.9,max_split_size_mb:512 call webui.bat

But I'm still getting the error message: "OutOfMemoryError: CUDA out of memory. Tried to allocate 44.00 MiB (GPU 0; 3.81 GiB total capacity; 2.87 GiB already allocated; 17.69 MiB free; 2.96 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Time taken: 5.0 sec. A: 2.88 GB, R: 2.96 GB, Sys: 3.8/3.8125 GB (99.9%)"

[StepHaze]

Try adding the "PYTORCH_CUDA_ALLOC_CONF" variable to the "webui-user.bat" file.

Example

Sampling steps are equal to 20. May it cause the problem?

[unLomTrois]

I've faced a similar error before, it happens because you don't have enough GPU memory (VRAM)

You have some options:

• disable hardware acceleration in your browser, so it will not use your GPU and all of it will be used by Stable Diffusion
• add --medvram to your COMMANDLINE_ARGS (or --lowvram)
• use token merging (you can find it in settings > optimizations), set it near 0.3
• add --xformers to your args if you have nvidia graphical card (or --opt-sdp-attention if not)
• also I'd recommend to use another browser for running automatic1111, because firefox cannot drag-n-drop images, which is very useful when you inpaint

You can see other performance-related command line args here: https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Command-Line-Arguments-and-Settings

I use something like this:

set COMMANDLINE_ARGS=--xformers --medvram

It works on my 4gb GPU

I hope it will help you

[unLomTrois]

This error may chase you if you want to make images larger than 1024x1024, unless you upgrade your GPU to 8-16gb

[StepHaze]

This error may chase you if you want to make images larger than 1024x1024, unless you upgrade your GPU to 8-16gb

I'm trying to make 512x512 images

[StepHaze]

• disable hardware acceleration in your browser

I'm using Firefox. Please show me where this option is.

[StepHaze]

• also I'd recommend to use another browser for running automatic1111

Which one should I use? Thanks!

[StepHaze]

You have some options:

I did everything you recommended, but still getting:

OutOfMemoryError: CUDA out of memory. Tried to allocate 16.00 MiB (GPU 0; 3.81 GiB total capacity; 2.92 GiB already allocated; 33.00 MiB free; 3.00 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

[K-Max-Me]

Do you have controlnet enabled? If so, make sure the low vram option is checked. You may have to turn off contronet completely.

Are you using a SD 1.5 model or a SD 2.1 model or a SDXL model? For 4gb, try using a SD 1.5 model since it will take the least amount of vram. SD 2.1 and SDXL models will take up way more vram.

How much system ram do you have?

[StepHaze]

How much system ram do you have?

8 GB. GPU has only 4 GB.

It seems modern neural networks are not for me((

[K-Max-Me]

You should also consider upgrading ram as well in general. At least to 16 GB.

[unLomTrois]

try --lowvram, maybe with --lowram