InvokeAI icon indicating copy to clipboard operation
InvokeAI copied to clipboard
verified publisher icon invoke-ai

[bug]: InvokeAI won't use GPU (CUDA/NVIDIA). Defaults to CPU

Open FalxGod opened this issue 3 years ago • 25 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues

OS

Windows

GPU

cuda

VRAM

24GB

What happened?

Starting invoke.bat is always saying that it uses the CPU instead of my GPU (And it indeed tries to generate with the CPU). Here's the cmd output with the relevant line indicated by an arrow. The line should read ">> Using device_type cuda" not "CPU"

1. command-line
2. browser-based UI
3. open the developer console
Please enter 1, 2 or 3: 2
Starting the InvokeAI browser-based UI..
* Initializing, be patient...
>> Initialization file C:\Users\Falxie II/.invokeai found. Loading...
>> InvokeAI runtime directory is "J:\AI\Art\StableDiffusion\invokeAI"
>> Patchmatch initialized
>> GFPGAN Initialized
>> CodeFormer Initialized
>> ESRGAN Initialized
>> Using device_type CPU     <-----------------------------
>> Current VRAM usage:  0.00G
>> Scanning Model: stable-diffusion-1.5
>> Model Scanned. OK!!
>> Loading stable-diffusion-1.5 from J:\AI\Art\StableDiffusion\invokeAI\models\ldm\stable-diffusion-v1\v1-5-pruned-emaonly.ckpt
   | LatentDiffusion: Running in eps-prediction mode
   | DiffusionWrapper has 859.52 M params.
   | Making attention of type 'vanilla' with 512 in_channels
   | Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
   | Making attention of type 'vanilla' with 512 in_channels
   | Using more accurate float32 precision
   | Loading VAE weights from: J:\AI\Art\StableDiffusion\invokeAI\models\ldm\stable-diffusion-v1\vae-ft-mse-840000-ema-pruned.ckpt
>> Model loaded in 7.44s
>> Current embedding manager terms: *
>> Setting Sampler to k_euler_a

* --web was specified, starting web server...
>> Initialization file C:\Users\Falxie II/.invokeai found. Loading...
>> Started Invoke AI Web Server!
>> Default host address now 127.0.0.1 (localhost). Use --host 0.0.0.0 to bind any address.
>> Point your browser at http://127.0.0.1:9090

Screenshots

Here's a screenshot of a prompt being executed. Notice the time to completion in the bottom right corner, grafik And this is my GPU during the generation process. Notice the utilization which also indicates that the process uses the CPU. grafik

Additional context

I reinstalled InvokeAI several times and it's always the same. Automatic1111 WebUI uses the GPU just fine.

Contact Details

No response

FalxGod avatar Dec 03 '22 19:12 FalxGod

When I added these 3 paths to my environment variables the source installer finally started working.

%SystemRoot%\system32 %SystemRoot% %SystemRoot%\System32\Wbem

also, make sure to check the paths in the small config file called .invokeai in the "user/'my_name'" folder. It seems to override your choices unless you edit the file. 1

vis223 avatar Dec 03 '22 19:12 vis223

When I added these 3 paths to my environment variables the source installer finally started working.

%SystemRoot%\system32 %SystemRoot% %SystemRoot%\System32\Wbem

also, make sure to check the paths in the small config file called .invokeai in the "user/'my_name'" folder. It seems to override your choices unless you edit the file. 1

I think you posted in the wrong thread? The installer works fine for me. And my .invokeai file doesn't cause any problems.

FalxGod avatar Dec 03 '22 19:12 FalxGod

@FalxGod Yes known issue with using CPU instead of GPU. Discord link with ideas of doing the manual install, and updating pytorch. https://discord.com/channels/1020123559063990373/1047568895168483348 InvokeAI 2.0.2 was using GPU and then switched to 2.0 and now uses CPU. So whatever changed in the version or if in that time pytorch made 11.6 the latest version? As of this post now known fix. Thread with more info on how to check/ install pytorch version: https://github.com/invoke-ai/InvokeAI/discussions/1628#discussioncomment-4277763

cybromancy avatar Dec 04 '22 00:12 cybromancy

@FalxGod pytorch doesnt work with python 3.11 and python doesn't like being deleted. Have to manually ctr+f 'python' in reg edit. Shows one entry delete then ctr+f again to find next.

here's from a clean install of python 3.10.8 pip install numpy

pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116

pip install virtualenv

virtualenv invokeai

cd c:/ to invokeAI folder (easiest way is to find invokeAI folder and right click the top address and copy) in cmd type cd ctr+v

copy environments-and-requirements\requirements-win-colab-cuda.txt requirements.txt

pip install --prefer-binary -r requirements.txt

python scripts/invoke.py --web

cybromancy avatar Dec 04 '22 05:12 cybromancy

@cybromancy Ty I will look into it! Imma leave the issue open for now.

FalxGod avatar Dec 04 '22 15:12 FalxGod

Still no luck here then :/ Damn.... I think this should be one of the main issues to look at. How many want to sun sirapy slow on a CPU when they can gp lightning fast on a GPU?!

Kallamamran avatar Dec 04 '22 21:12 Kallamamran

I am having the same issue. RTX 3080 Ti. Its using CPU instead of GPU. This is from a fresh automated install.

Xerophayze avatar Dec 04 '22 22:12 Xerophayze

Yeah I’m not going to even bother trying at this point.

DivinAria avatar Dec 04 '22 23:12 DivinAria

Still no luck here then :/ Damn.... I think this should be one of the main issues to look at. How many want to sun sirapy slow on a CPU when they can gp lightning fast on a GPU?!

It is 100% the main issue. I wish I’d never come across this app tbh it’s been nothing but a tease and a headache.

DivinAria avatar Dec 04 '22 23:12 DivinAria

EASY FIX:

  1. Open enviroment.yml
  2. change line 9,10,11 to
  • torchvision=0.13.1+cu116
  • torchaudio=0.12.1+cu116
  • pytorch=1.12.1+cu116
  1. run install.bat
  2. run invoke.bat grafik Running cuda now

FalxGod avatar Dec 05 '22 20:12 FalxGod

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

  1. Open enviroment.yml
  2. change line 9,10,11 to
  • torchvision=0.13.1+cu116
  • torchaudio=0.12.1+cu116
  • pytorch=1.12.1+cu116
  1. run install.bat
  2. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

Xerophayze avatar Dec 05 '22 21:12 Xerophayze

somehow it revered that yml file back... ill try again.

On Mon, Dec 5, 2022 at 2:30 PM Echo T @.***> wrote:

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

  1. Open enviroment.yml
  2. change line 9,10,11 to
  • torchvision=0.13.1+cu116
  • torchaudio=0.12.1+cu116
  • pytorch=1.12.1+cu116
  1. run install.bat
  2. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

Xerophayze avatar Dec 05 '22 21:12 Xerophayze

so when I run the install.bat after making the changes to the yml file, it redownloads the file, removing the changes i just made.

On Mon, Dec 5, 2022 at 2:32 PM Echo T @.***> wrote:

somehow it revered that yml file back... ill try again.

On Mon, Dec 5, 2022 at 2:30 PM Echo T @.***> wrote:

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

  1. Open enviroment.yml
  2. change line 9,10,11 to
  • torchvision=0.13.1+cu116
  • torchaudio=0.12.1+cu116
  • pytorch=1.12.1+cu116
  1. run install.bat
  2. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

Xerophayze avatar Dec 05 '22 21:12 Xerophayze

I changed the file to read only so it could not replace the file, but this is what it shows. [image: image.png]

On Mon, Dec 5, 2022 at 2:38 PM Echo T @.***> wrote:

so when I run the install.bat after making the changes to the yml file, it redownloads the file, removing the changes i just made.

On Mon, Dec 5, 2022 at 2:32 PM Echo T @.***> wrote:

somehow it revered that yml file back... ill try again.

On Mon, Dec 5, 2022 at 2:30 PM Echo T @.***> wrote:

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

  1. Open enviroment.yml
  2. change line 9,10,11 to
  • torchvision=0.13.1+cu116
  • torchaudio=0.12.1+cu116
  • pytorch=1.12.1+cu116
  1. run install.bat
  2. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

Xerophayze avatar Dec 05 '22 21:12 Xerophayze

another thing, in the file its not line 9,10,11. its 10,11,12.

[image: image.png]

On Mon, Dec 5, 2022 at 2:40 PM Echo T @.***> wrote:

I changed the file to read only so it could not replace the file, but this is what it shows. [image: image.png]

On Mon, Dec 5, 2022 at 2:38 PM Echo T @.***> wrote:

so when I run the install.bat after making the changes to the yml file, it redownloads the file, removing the changes i just made.

On Mon, Dec 5, 2022 at 2:32 PM Echo T @.***> wrote:

somehow it revered that yml file back... ill try again.

On Mon, Dec 5, 2022 at 2:30 PM Echo T @.***> wrote:

did all that and it still says CPU [image: image.png]

On Mon, Dec 5, 2022 at 1:40 PM Falxie @.***> wrote:

EASY FIX:

  1. Open enviroment.yml
  2. change line 9,10,11 to
  • torchvision=0.13.1+cu116
  • torchaudio=0.12.1+cu116
  • pytorch=1.12.1+cu116
  1. run install.bat
  2. run invoke.bat [image: grafik] https://user-images.githubusercontent.com/39287404/205738730-f7b58c87-b5b9-4d1b-a242-47da3ba1d3ab.png Running cuda now

— Reply to this email directly, view it on GitHub https://github.com/invoke-ai/InvokeAI/issues/1763#issuecomment-1338136720, or unsubscribe https://github.com/notifications/unsubscribe-auth/A3BHMCA2XIHTD7NIUNOYEW3WLZHNNANCNFSM6AAAAAASS4ZXEU . You are receiving this because you commented.Message ID: @.***>

Xerophayze avatar Dec 05 '22 21:12 Xerophayze

Here's what I have done in total before it worked for me (excerpt from discord): I also deleted the C.\Users[username].invokeai file before running install.bat and invoke.bat (after editing the lines; which I just realized by reading your comment and another in the github issue did nothing because it gets overwritten by running install.bat)

Im doubtful it will help, because the .invoke file says nothing about cuda...

I also did some steps from the manual install but not all. I ran:

  1. git clone https://github.com/invoke-ai/InvokeAI.git
  2. conda env update
  3. conda activate invokeai
  4. python scripts/invoke.py

Then I deleted the whole folder again. (Because I got lazy~~) After that I ran these 2 commands in my cmd:

  1. pip install numpy
  2. pip install torch==1.12.1+cu116 torchvision==0.13.1+cu116 torchaudio==0.12.1 --extra-index-url https://download.pytorch.org/whl/cu116

After that i edited the 3 lines in the enviroment.yml and it worked

FalxGod avatar Dec 06 '22 07:12 FalxGod

Yeah that didn't work (W10) image

Kallamamran avatar Dec 06 '22 09:12 Kallamamran

2. pip install torch==1.12.1+cu116

That didn't work either image

Kallamamran avatar Dec 06 '22 09:12 Kallamamran

@Kallamamran what version of Python do you have installed? edit: If you have Python 11 then uninstall it and get python 10.x. PyTorch is not supported in version 11 of Python at the moment. (Had the same error when using Python 11)

FalxGod avatar Dec 06 '22 10:12 FalxGod

I Have installed invokeai 2.5.5 Completely Without error Now when i start invokeai.bat This appears

  1. command-line
  2. browser-based UI
  3. open the developer console Please enter 1, 2 or 3 Now When i type 1 or 2 it won't work only typing 3 is working how to fix this

nazeerPro avatar Dec 13 '22 03:12 nazeerPro

When you are looking at the Windows performance monitor, make sure you've expanded the details and changed one of the graphs to Cuda. The default load % shown could be for the 3D pipeline, not Cuda.

So when you're looking at if a patch worked or not, you want to see the Cuda load percentage, like in this example from my Windows 10 machine:

image

VimCommando avatar Dec 18 '22 20:12 VimCommando

@VimCommando When you start InvokeAI, Is one of the first lines >> Patchmatch initialized?

I am just currious, since I have the feeling that the precompiled dll for windows is only compiled with CPU Features. But this should of course not stop the other Modules from accessing the GPU (which is why it would be great if you have Patchmatch initialized 😅)

EDIT:

Just saw that we already have prove that patchmatch does not block the use of cuda:

grafik

mauwii avatar Dec 23 '22 14:12 mauwii

Plot twist: my above statement is only true if hardware accelerated GPU scheduling is not enabled: https://devblogs.microsoft.com/directx/hardware-accelerated-gpu-scheduling/

Once you turn this on, then CUDA is no longer a separate category for the performance monitor, and the load reflects in the main utilization graph as expected.

image

I discovered this after upgrading my graphics card and trying to get a completely unrelated program running.

VimCommando avatar Jan 10 '23 05:01 VimCommando

I fixed it with:

pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 -f https://download.pytorch.org/whl/torch_stable.html

(while in the environment)

You can check if cuda is working with:

$ python
Python 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True

After that, I directly ran invoke.py.

rafalou38 avatar Jan 27 '23 21:01 rafalou38

I fixed it with:

pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 -f https://download.pytorch.org/whl/torch_stable.html

(while in the environment)

You can check if cuda is working with:

$ python
Python 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True

After that, I directly ran invoke.py.

Yup this worked for me!!. i used invoke.bat image

veetance avatar Feb 01 '23 10:02 veetance

The above fix didn't work for me but this similar one did: https://github.com/invoke-ai/InvokeAI/issues/2460#issuecomment-1416056021

SeanBannister avatar Oct 10 '23 16:10 SeanBannister