stable-diffusion-webui icon indicating copy to clipboard operation
stable-diffusion-webui copied to clipboard

[Bug]: GPU clock and memory clock are full, GPU load 0, before painting

Open woshitoutouge opened this issue 1 year ago • 5 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues and checked the recent builds/commits

What happened?

GPU frequency and memory frequency are full, GPU load is 0, but i haven't started painting yet.

image

image

image

It’s not like this every time. Sometimes after starting the service, the GPU frequency, memory frequency, and graphics card power consumption are low.

Steps to reproduce the problem

  1. run webui-user.bat
  2. check GPU-Z

What should have happened?

low GPU clock and Memory clock if no painting

Commit where the problem happens

commit: 48a15821

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

--xformers

List of extensions

sd-webui-controlnet

clip-interrogator-ext

Console logs

venv "D:\Code\webui\venv\Scripts\Python.exe"
Python 3.10.9 | packaged by conda-forge | (main, Jan 11 2023, 15:15:40) [MSC v.1916 64 bit (AMD64)]
Commit hash: 48a15821de768fea76e66f26df83df3fddf18f4b
Installing requirements for Web UI


Launching Web UI with arguments: --xformers
D:\Code\webui\venv\lib\site-packages\torchvision\transforms\functional_tensor.py:5: UserWarning: The torchvision.transforms.functional_tensor module is deprecated in 0.15 and will be **removed in 0.17**. Please don't rely on it. You probably just need to use APIs in torchvision.transforms.functional or in torchvision.transforms.v2.functional.
  warnings.warn(
Loading config from: D:\Code\webui\models\Stable-diffusion\v1-5-pruned.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading weights [1a189f0be6] from D:\Code\webui\models\Stable-diffusion\v1-5-pruned.safetensors
Loading VAE weights specified in settings: D:\Code\webui\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(0):
Model loaded in 8.7s (0.5s create model, 5.9s load weights).
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.

Additional information

No response

woshitoutouge avatar Apr 11 '23 01:04 woshitoutouge

Same bug here, the GPU AND CPU clocks remain high after starting. Here consuming around 50 watts for nothing on a 2070 RTX until generating the first picture.

To devs: easy method to reproduce the bug -> Load webui-user as usual but with an empty \embeddings\ folder, or just without any valid file inside. Console should say it loaded 0, like in the OP log: Textual inversion embeddings loaded(0): and the bug should happen. (you can check your CPU and GPU speeds and power consumption with dedicated tools like GPUZ or HWiNFO64)

To users: a workaround to prevent this bug to happen -> ensure at least one valid textual inversion embeddings file is installed in the embeddings\ folder. Then few seconds later after starting the GPU and CPU clocks should return to normal state.

byblo avatar Apr 12 '23 09:04 byblo

@byblo I've tried it but I'm not sure if it worked, because the video card state is not always normal. I found that if GPU clock and power consume are high, just close the reminal and restart, then it become normal. Thanks anyway!

image

woshitoutouge avatar Apr 13 '23 07:04 woshitoutouge

Weird. Here it always get fixed the way I explained.

Just to be sure, can you try to run it and wait until the console message is visible: 'To create a public link, etc...', then wait like 1 minute more then check again the clock speeds?

On my side, I just need to wait like 30 seconds after webui has fully loaded, then the clocks dropping again to idle (400Mhz and 300Mhz), and the power consumption drops from 50 to 12 watts.

byblo avatar Apr 13 '23 12:04 byblo

@byblo My situation here is a bit different. As soon as it starts up, it works normally without waiting.

image

Good news is after i add a textual file, every time the webUI restarts, the frequency and power consumption are normal. It’s just sometimes don’t know how, but the frequency won't decrease.

woshitoutouge avatar Apr 13 '23 13:04 woshitoutouge

I'll keep an eye on my side and check clocks each time i restart the webui.

byblo avatar Apr 13 '23 14:04 byblo

The bug hasn’t reappeared for a long time, perhaps it was resolved by updating the GPU driver.

woshitoutouge avatar Aug 05 '23 01:08 woshitoutouge