InvokeAI [bug]:

Is there an existing issue for this?

[X] I have searched the existing issues

OS

Linux

GPU

cuda

VRAM

nvidia-smi --query-gpu=memory.total --format=csv,noheader 1999 MiB

What happened?

So I recently got my Nvidia up and running in Debian 11, however now because InvokeAI is trying to use CUDA I'm getting the following error:

File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/torch/nn/modules/module.py", line 987, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 74.00 MiB (GPU 0; 1.95 GiB total capacity; 1.21 GiB already allocated; 21.38 MiB free; 1.26 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Here is an entire dump from starting --web

A matching Triton is not available, some optimizations will not be enabled. Error caught was: No module named 'triton'

Initializing, be patient...

Initialization file /home/ep/InvokeAI/invokeai.init found. Loading... Internet connectivity is True InvokeAI, version 2.3.0 InvokeAI runtime directory is "/home/ep/InvokeAI" GFPGAN Initialized CodeFormer Initialized ESRGAN Initialized Using device_type cuda xformers memory-efficient attention is available and enabled Current VRAM usage: 0.00G Loading diffusers model from runwayml/stable-diffusion-v1-5 | Using faster float16 precision | Loading diffusers VAE from stabilityai/sd-vae-ft-mse Fetching 15 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 495.38it/s] ** model stable-diffusion-1.5 could not be loaded: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 1.95 GiB total capacity; 1.98 MiB already allocated; 1.12 MiB free; 2.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF Traceback (most recent call last): File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/ldm/generate.py", line 889, in set_model model_data = cache.get_model(model_name) File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/ldm/invoke/model_manager.py", line 106, in get_model requested_model, width, height, hash = self._load_model(model_name) File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/ldm/invoke/model_manager.py", line 341, in _load_model model, width, height, model_hash = self._load_diffusers_model(mconfig) File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/ldm/invoke/model_manager.py", line 532, in _load_diffusers_model pipeline.to(self.device) File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/diffusers/pipelines/pipeline_utils.py", line 272, in to module.to(torch_device) File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/torch/nn/modules/module.py", line 989, in to return self._apply(convert) File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/torch/nn/modules/module.py", line 641, in _apply module._apply(fn) File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/torch/nn/modules/module.py", line 641, in _apply module._apply(fn) File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/torch/nn/modules/module.py", line 641, in _apply module._apply(fn) [Previous line repeated 3 more times] File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/torch/nn/modules/module.py", line 664, in _apply param_applied = fn(param) File "/home/ep/InvokeAI/invokeai/lib/python3.9/site-packages/torch/nn/modules/module.py", line 987, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 1.95 GiB total capacity; 1.98 MiB already allocated; 1.12 MiB free; 2.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

** trying to reload previous model

Loading diffusers model from runwayml/stable-diffusion-v1-5 | Using faster float16 precision | Loading diffusers VAE from stabilityai/sd-vae-ft-mse Fetching 15 files: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 15/15 [00:00<00:00, 19636.25it/s] ** An error occurred while attempting to initialize the model: "CUDA out of memory. Tried to allocate 2.00 MiB (GPU 0; 1.95 GiB total capacity; 1.99 MiB already allocated; 1.12 MiB free; 2.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF" ** This can be caused by a missing or corrupted models file, and can sometimes be fixed by (re)installing the models.

** This can be caused by a missing or corrupted models file, and can sometimes be fixed by (re)installing the models. Do you want to run invokeai-configure script to select and/or reinstall models? [y] n

--web was specified, starting web server...
Initializing, be patient...

Initialization file /home/ep/InvokeAI/invokeai.init found. Loading... Started Invoke AI Web Server! Default host address now 127.0.0.1 (localhost). Use --host 0.0.0.0 to bind any address. Point your browser at http://127.0.0.1:9090 System config requested Patchmatch initialized

Opening the Web UI - I see GREEN Connected Dreamlike-diffusion-1.0

Trying to generate produces the following error

torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 1.95 GiB total capacity; 24.36 MiB already allocated; 512.00 KiB free; 26.00 MiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

I can't seem to load any models, ger ERROR in the UI when I do - I know the models are installed, have downloaded multiple times and it was working prior in 2.3.0+a0 - Now after "upgrade" new UI with negative prompt says 2.3.0 weird

Has anyone gotten further - I can see a ton of people asking about very similar issues

Operating System: Debian GNU/Linux 11 KDE Plasma Version: 5.20.5 KDE Frameworks Version: 5.78.0 Qt Version: 5.15.2 Kernel Version: 5.10.0-21-amd64 OS Type: 64-bit Processors: 8 × Intel® Core™ i7-4790K CPU @ 4.00GHz Memory: 15.6 GiB of RAM Graphics Processor: NVIDIA GeForce GTX 750 Ti/PCIe/SSE2

System free -h total used free shared buff/cache available Mem: 15Gi 2.1Gi 9.8Gi 39Mi 3.7Gi 13Gi Swap: 975Mi 20Mi 955Mi

GPU:

nvidia-smi --query-gpu=memory.total --format=csv,noheader 1999 MiB

+-----------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=============================================================================| | 0 N/A N/A 843 G /usr/lib/xorg/Xorg 129MiB | | 0 N/A N/A 1392 G /usr/bin/kwin_x11 29MiB | | 0 N/A N/A 1460 G /usr/bin/plasmashell 31MiB | | 0 N/A N/A 1564 G ...b/firefox-esr/firefox-esr 119MiB | | 0 N/A N/A 3189 G /usr/bin/systemsettings5 25MiB | +-----------------------------------------------------------------------------+

Screenshots

No response

Additional context

No response

Contact Details

No response

Feb 16 '23 17:02 einarpetersen

A user said: According to that, your GPU falls under the requirements for InvokeAI - check them out here: https://invoke-ai.github.io/InvokeAI/#hardware-requirements

— Today at 6:37 PM Works perfectly just using the CPU - Why would adding GPU make it fail - But yeah I see the recommendations.... hmmm Or I should perhaps say used to work 🙂

— Today at 6:48 PM I will see if I can downgrade to CPU again and if it will work in the new UI Einar Petersen (Comics Creator)

— Today at 7:22 PM did not help to do the : pip install InvokeAI --use-pep517 --extra-index-url https://download.pytorch.org/whl/cpu install.... still getting File "/home/ep/InvokeAI/.venv/lib/python3.9/site-packages/torch/nn/modules/module.py", line 987, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 74.00 MiB (GPU 0; 1.95 GiB total capacity; 1.24 GiB already allocated; 6.06 MiB free; 1.28 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF

Feb 16 '23 18:02 einarpetersen

So somehow InvokeAI still tries to use the CUDA device despite it being installed as CPU only...

Feb 16 '23 18:02 einarpetersen

An easy fix might be export CUDA_VISIBLE_DEVICES=-1 (set in your shell environment as persistently as you like)
you could try to pip uninstall torch and then pip install torch --force-reinstall --extra-index-url https://download.pytorch.org/whl/cpu. This should ensure you are using the CPU version of torch.
A "nuclear" option might be to remove your InvokeAI virtual environment completely, purge all the pip caches, and try to reinstall again to make sure you're not getting a GPU torch from somewhere.

But if (1) works, that might be all that's needed.

Feb 20 '23 08:02 ebr

Ended up doing a nuclear - I can generate images again, however masking/in-painting no longer works....

Feb 21 '23 21:02 einarpetersen

As in no mask appears no matter brush color picked layer chosen - Simply non existent - eraser has also disappeared

Feb 21 '23 21:02 einarpetersen

no mask appears no matter brush color picked layer chosen

This seems to be a UI issue; couldn't reproduce it on my end. Could you please try the Reset Web UI button under Settings and see if it helps?

Mar 03 '23 16:03 ebr

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

Mar 18 '23 06:03 github-actions[bot]

Did a pip upgrade and then inpaint etc. started working again

Mar 18 '23 14:03 einarpetersen

There has been no activity in this issue for 14 days. If this issue is still being experienced, please reply with an updated confirmation that the issue is still being experienced with the latest release.

Apr 03 '23 06:04 github-actions[bot]