[Bug]: Unable to select specific GPU in multi-GPU system

[ooofest]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

In a dual-GPU Windows 10 system (2 x RTX-3090), only the first GPU is being used by txt2img processing, despite specifying that SD should only use the second GPU.

Steps to reproduce the problem

1. Add "SET CUDA_VISIBLE_DEVICES=1" to webui-user.bat
2. Run webui-user.bat, wait for the local process to initialize
3. Access https://localhost:7860/
4. Execute any prompt + configuration + model combination in txt2img
5. View Task Manager -> Performance tab and note that only GPU 0 is being used + VRAM allocated by SD (alternatively, check GPU usage in ASUS GPU Monitor III, with the same result)

What should have happened?

In step (5.) only GPU 1 should reflect txt2img processing + memory allocation

Commit where the problem happens

572f0aa2b6

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Mozilla Firefox

Command Line Arguments

Contents of webui-user.bat:

set PYTHON=
set GIT=
set VENV_DIR=

set COMMANDLINE_ARGS=--xformers
set SET CUDA_VISIBLE_DEVICES=1
set SAFETENSORS_FAST_GPU=1

call webui.bat

List of extensions

N/A

Console logs

venv "D:\A1111-stable-diffusion\venv\Scripts\Python.exe"
Python 3.10.8 (tags/v3.10.8:aaaf517, Oct 11 2022, 16:50:30) [MSC v.1933 64 bit (AMD64)]
Commit hash: <none>
Installing requirements for Web UI
Launching Web UI with arguments: --xformers
Loading weights [e1441589a6] from D:\A1111-stable-diffusion\models\Stable-diffusion\sd-v1-5-pruned.ckpt
Creating model from config: D:\A1111-stable-diffusion\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading VAE weights specified in settings: D:\A1111-stable-diffusion\models\Stable-diffusion\sd-v1-5-pruned-emaonly.vae.pt
Applying xformers cross attention optimization.
Model loaded in 4.2s (load weights from disk: 1.5s, create model: 0.3s, apply weights to model: 0.4s, apply half(): 0.5s, load VAE: 0.1s, move model to device: 0.4s).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
100%|██████████████████████████████████████████████████████████████████████████████████| 90/90 [00:06<00:00, 13.75it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 90/90 [00:05<00:00, 16.00it/s]

Additional information

Specifying use of GPU 1 had worked for some months, it seems that GPU 0 was being used for only recent builds (the past month or so.)

[vladmandic]

if you're using accelerate (which is enabled by default), you may want to run accelerate config once to setup your config.

[ooofest]

if you're using accelerate (which is enabled by default), you may want to run accelerate config once to setup your config.

Thanks, I had done that a long time ago, but wasn't SET CUDA_VISIBLE_DEVICES supposed to override the accelerate config? At least in past usage, that seemed to work when I switched between GPUS (depending on which was being used by other programs at the time.)

Nonetheless, I re-ran accelerate config and selected the second GPU, but the code still used my first GPU, unfortunately.

Tried erasing everything and installing earlier commits but the same lack of GPU choice is happening.

And yet, I can easily choose the GPU in other programs. So, this is odd. I wonder if this is at all related to torch level.

[ooofest]

User error!

The first post shows my problem - probably a copy/paste issue where it specified:

set SET CUDA_VISIBLE_DEVICES=1

instead of

set CUDA_VISIBLE_DEVICES=1

The latter environment variable setting works as expected.

[Panchovix]

Thanks, I just connected a 2nd 4090 on my PC today and had the same issue, appreciated!

[go-noah]

I solved this problem I will submit a PR with the improved code