Torch is not able to use GPU

[colossalsthegame]

First there were issues with the torch hash code, and now it says torch is unable to use GPU.

venv "C:\stable-diffusion-webui-master\stable-diffusion-webui-master\venv\Scripts\Python.exe" Python 3.10.7 (tags/v3.10.7:6cc6b13, Sep 5 2022, 14:08:36) [MSC v.1933 64 bit (AMD64)] Commit hash: Traceback (most recent call last): File "C:\stable-diffusion-webui-master\stable-diffusion-webui-master\launch.py", line 98, in run_python("import torch; assert torch.cuda.is_available(), 'Torch is not able to use GPU'") File "C:\stable-diffusion-webui-master\stable-diffusion-webui-master\launch.py", line 50, in run_python return run(f'"{python}" -c "{code}"', desc, errdesc) File "C:\stable-diffusion-webui-master\stable-diffusion-webui-master\launch.py", line 44, in run raise RuntimeError(message) RuntimeError: Error running command. Command: "C:\stable-diffusion-webui-master\stable-diffusion-webui-master\venv\Scripts\python.exe" -c "import torch; assert torch.cuda.is_available(), 'Torch is not able to use GPU'" Error code: 1 stdout: stderr: Traceback (most recent call last): File "", line 1, in AssertionError: Torch is not able to use GPU

Press any key to continue . . .

[ixateht]

I am seeing this as well. I am also seeing the following error 3 times at the beginning of running the batch file.

"The filename, directory name, or volume label syntax is incorrect."

I'm not sure if it is related.

[JelleBouma]

I have the same error message using Python 3.10.6 [MSC v.1932 64 bit (AMD64)] with RX 6750 XT I don't get the incorrect filename error message though.

[greyfoxone]

Same issue here

[ChinatsuHS]

get the error mostly when using non-default clocks on the used GPU ... try resetting overclocked gpu profiles to default should fix it

[quickwick]

I'm getting the same error.

• Windows 10 Pro 21H2
• Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
• AMD Radeon RX 580 GPU

I looked up ROCm Supported GPUs. According to what I found, this GPU is "enabled in the ROCm software, through full support is not guaranteed"

I don't think the specific model of AMD GPU is important, through. More likely it has something to do with assert torch.cuda.is_available. CUDA is, of course, NVIDIA-specific.

[ixateht]

The "The filename, directory name, or volume label syntax is incorrect." error for me was incorrectly commented out lines in the webui-user.bat file.

I reverted back to before the previous 'git pull' with a 'git reset --hard HEAD@{1}' and the "Torch is not able to use GPU" issue went away so it is/was something in a code change and not a local issue.

[quickwick]

I reverted back to before the previous 'git pull' with a 'git reset --hard HEAD@{1}' and the "Torch is not able to use GPU" issue went away so it is/was something in a code change and not a local issue.

Does everything else work for you after doing this? Can you generate images in the webui?

It looks like the install guide for AMD GPUs (https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-AMD-GPUs) was updated in the last 16 hours, to include a link to an untested external guide for getting AMD GPUs working on Windows. Previous to that, the AMD GPUs install guide made no mention of Windows.

[name-changer]

Radeon 6600XT on Fedora 36 here. The error came up on my system, too.

It now works for me after following the previously mentioned install guide for AMD GPUs and starting the UI with HSA_OVERRIDE_GFX_VERSION=10.3.0 TORCH_COMMAND='pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/rocm5.1.1' python launch.py

The flags --precision full --no-half crash the application for me. As mentioned in the install guide, the first generation did take a while to complete, but afterwards it all seems to works fine.

[Saturnix]

I reverted back to before the previous 'git pull' with a 'git reset --hard HEAD@{1}' and the "Torch is not able to use GPU" issue went away so it is/was something in a code change and not a local issue.

Does everything else work for you after doing this? Can you generate images in the webui?

It looks like the install guide for AMD GPUs (https://github.com/AUTOMATIC1111/stable-diffusion-webui/wiki/Install-and-Run-on-AMD-GPUs) was updated in the last 16 hours, to include a link to an untested external guide for getting AMD GPUs working on Windows. Previous to that, the AMD GPUs install guide made no mention of Windows.

No idea why this guide is flagged as Linux only, with an external guide for Windows. The external guide didn't work for me, as apparently 8GB RAM are not enough for Onnx.

The Linux guide worked well when applied to Windows. For those in my situation and not tech-savy enough to translate it from Linux to Windows, here's how.

0. This assumes model.ckpt is already in the models folder and you've already git cloned the repository.

1. Delete the venv and tmp folders, if they're present.

2. Open a command prompt, cd into the main stable-diffusion-webui folder, and type:

python -m venv venv venv/Scripts/activate.bat python -m pip install --upgrade pip wheel pip install torch torchvision --extra-index-url https://download.pytorch.org/whl/rocm5.1.1

3. Open the file launch.py with a text editor. Look for if not skip_torch_cuda_test and False: (currently at line 110), and replace it with if False:. Save the file.

4. python launch.py --precision full --no-half

To launch again, you only need tu run activate.bat and step 4.

This works fine on Win10, 8GB RAM and an RX580 8GB Vram, although is 22 times slower than what I get with Nvidia 3060.

[quickwick]

@Saturnix You're able to generate images successfully?

I tried following your steps. When running launch.py, I receive the error: Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled Eventually it tells me that it's running on local URL. I can open the web UI, but if I try to generate anything, I get the error AssertionError: Torch not compiled with CUDA enabled

Here's something interesting. If you go to https://pytorch.org/get-started/locally/ and select Stable (or LTS), Windows, Pip, Python... the ROCm5.1.1 compute platform is crossed out and there's the message NOTE: ROCm is not available on Windows.

[Saturnix]

Yeah, I’ve noticed that but, strangely enough, it did work. I was able to generate an image.

Which GPU and Python version are you running?

---

From: quickwick @.> Sent: Tuesday, September 27, 2022 06:58 To: AUTOMATIC1111/stable-diffusion-webui @.> Cc: Saturnix @.>; Mention @.> Subject: Re: [AUTOMATIC1111/stable-diffusion-webui] Torch is not able to use GPU (Issue #759)

@Saturnixhttps://github.com/Saturnix You're able to generate images successfully?

I tried following your steps. When running launch.py, I receive the error: Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled Eventually it tells me that it's running on local URL. I can open the web UI, but if I try to generate anything, I get the error AssertionError: Torch not compiled with CUDA enabled

Here's something interesting. If you go to https://pytorch.org/get-started/locally/ and select Stable (or LTS), Windows, Pip, Python... the ROCm5.1.1 compute platform is crossed out and there's the message NOTE: ROCm is not available on Windows.

— Reply to this email directly, view it on GitHubhttps://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/759#issuecomment-1258902595, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AALYRDXRPATHXQGOQHGB6MTWAJPEVANCNFSM6AAAAAAQRGOVXI. You are receiving this because you were mentioned.Message ID: @.***>

[quickwick]

• Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug 1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
• AMD Radeon RX 580 8GB GPU

[colossalsthegame]

@Saturnix You're able to generate images successfully?

I tried following your steps. When running launch.py, I receive the error: Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled Eventually it tells me that it's running on local URL. I can open the web UI, but if I try to generate anything, I get the error AssertionError: Torch not compiled with CUDA enabled

Here's something interesting. If you go to https://pytorch.org/get-started/locally/ and select Stable (or LTS), Windows, Pip, Python... the ROCm5.1.1 compute platform is crossed out and there's the message NOTE: ROCm is not available on Windows.

I'm getting the same problem.

Warning: caught exception 'Torch not compiled with CUDA enabled', memory monitor disabled LatentDiffusion: Running in eps-prediction mode

And then there are a lot of errors in eps-prediction mode...

[quickwick]

@Saturnix are you perhaps somehow running in CPU mode? I've done a bunch of googling, and everything I can find says ROCm is not available for Windows.

[Saturnix]

are you perhaps somehow running in CPU mode?

Double checked and yes. No idea why or how, but by doing what I've told above it is actually working, but GPU has 0% utilization. All the load goes on the CPU (Ryzen 5 1600).

For those of you looking to use the GPU, the Windows guide (linked inside the Linux guide) is the best bet although, as I've said, I didn't manage to make it work, due to low system RAM (8GB).

At this point, I'm thinking maybe you guys are running Intel CPUs? What would be the difference anyway? No other idea otherwise: just git cloned the repo, venved a default venv, pip installed with --extra-index-url, removed the skip_torch_cuda_test in line 110 and then launched.py with listed parameters.

If you did these steps, it should run for you too, unless I'm a moron and skipped something. I did completely miss the fact that my GPU wasn't running, so there's that 😅

The repo might have been slightly tampered with, as this wasn't my first try... but I did delete the venv and tmp folders, so that should be like a brand newly cloned repo. Might try to re-do everything from scratch if that's helpful.

[quickwick]

For those of you looking to use the GPU, the Windows guide (linked inside the Linux guide) is the best bet although, as I've said, I didn't manage to make it work, due to low system RAM (8GB).

I have the Onnx/DirectML method working (16GB system RAM). I've even built a basic Tkinter GUI. I was just hoping to get this webui working, as it has so many extra options/features. Ah well.

At this point, I'm thinking maybe you guys are running Intel CPUs? What would be the difference anyway? No other idea otherwise: just git cloned the repo, venved a default venv, pip installed with --extra-index-url, removed the skip_torch_cuda_test in line 110 and then launched.py with listed parameters.

Yup, Intel CPU here.

Even if I could get this webui working in CPU mode, I don't think I'd bother. Stable Diffusion is already slow enough using GPU-powered Onnx/DirectML. You said CPU-based is 22x slower than NVIDIA powered, that sounds awful.

[Saturnix]

Even if I could get this webui working in CPU mode, I don't think I'd bother. Stable Diffusion is already slow enough using GPU-powered Onnx/DirectML. You said CPU-based is 22x slower than NVIDIA powered, that sounds awful.

Indeed it is.

It was just a test to see if I could run it on my desktop, and I've shared how I did it only for people who don't have an alternative.

The 3060 on my laptop is the only actually usable option. But if I didn't have that, I'd either pay for a cloud service or suffer locally at 3/4min per image.

EDIT

Just a heads up: other people are trying the Linux guide and report that the AMD GPU remains indeed untouched. Linux guide goes to CPU mode, for some reason.

See issue https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1220

[JABirchall]

Their are 2 main issues with getting it working in amd is the

1. miss-infomation spread by "tutorials". They all say you meed rocm to be able to use AMD, Wrong, AMD has even said them self on the github repo for a reason why rocm isnt avaible for windows is because rocm is for distributed OpenCL compute over many systems And windows is terrible at this. For a single system, the amdgpu drivers are fine. as all AMD GPUs have openCL compute enabled.

2. Developers selfishly developing for a closed system, just like nvidia intends. Despite nvidia cards being full compatible with OpenCL, they still push and succeed with getting developers to develop for there closed off platforms.

Because developers are selfish you have to use a differnct project: https://github.com/harishanand95/diffusers supoorts both AMD and Nvida (and prolly intel gpus too)

[quickwick]

Because developers are selfish you have to use a differnct project: https://github.com/harishanand95/diffusers supoorts both AMD and Nvida (and prolly intel gpus too)

You don't actually need to use the harishanand95 fork. It was forked from huggingface/diffusers repo and hasn't been updated since Aug 31. Sometime in early September, it looks like the fixes/changes were merged into the main huggingface/diffusers repo. So you can just install diffusers v0.3.0 and it works on AMD.

Here's the guide I followed to get Stable Diffusion working in Windows with AMD GPU: https://www.travelneil.com/stable-diffusion-windows-amd.html#b0e4230b-fc8b-4aec-b5bc-fb67784d78e2

And I might as well plug the janky little Tkinter UI I wrote: https://github.com/quickwick/stable-diffusion-win-amd-ui

[Supernaturalheadshot]

are you perhaps somehow running in CPU mode?

Double checked and yes. No idea why or how, but by doing what I've told above it is actually working, but GPU has 0% utilization. All the load goes on the CPU (Ryzen 5 1600).

Windows uses 'power saving' mode by default, which causes it to use your CPU instead of your GPU. You can go into settings and change it to 'high performance' as default. I believe installing the Cuda toolkit should also change it to only use GPU. In case someone has a similar issue and finds this thread, I just wanted to mention that this setting could be causing it to continuously try to use CPU instead of your GPU and cause the "torch cannot access GPU" error.

[JABirchall]

Windows uses 'power saving' mode by default, which causes it to use your CPU instead of your GPU. You can go into settings and change it to 'high performance' as default. I believe installing the Cuda toolkit should also change it to only use GPU. In case someone has a similar issue and finds this thread, I just wanted to mention that this setting could be causing it to continuously try to use CPU instead of your GPU and cause the "torch cannot access GPU" error.

The issue here is lack of support for AMD and Intel GPUs

[RHeynsZa]

On Ubuntu, I had this issue when running an AMD CPU with onboard graphics, and a Nvidia GPU with CUDA 11.7

My fix for this was:

• Run the initial webui.sh
• Do pip list
• See torch and torchvision installed with +rocm5.2
• Uninstall those versions with pip uninstall
• Get CUDA version via nvidia-smi
• Install torch version following instructions here
• Do a quick sanity check via pip3 list
• ????
• Profit

My educated guess would be that the script installs for the current enabled graphics by default. This is why people have luck when running in performance mode, since that switches from CPU graphics (lower power consumption) to GPU graphics. As @Supernaturalheadshot mentions

This definitely seems like a bug though @mezotaken , can you please add the label back?

[ClashSAN]

this is an old issue which is too generic, if this issue is related to future issues, please refer to this previous one.