sd-webui-controlnet
sd-webui-controlnet copied to clipboard
Mikubill
[Bug]: Whole WebUI is stuck when ControlNet is enabled.
Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui
What happened?
ControlNet is enabled thru the whole process in the Extensions, BUT it is not used to generate. Yet somehow managing to constantly break my webui. It is not random tho, every time I try to reproduce that error its happening.
Steps to reproduce the problem
1 Generated image with txt2img 2 Processed the finished img with img2img with custom parametres. 3 Did SwinIRx4 upscale with scale 2 4 Load the finished image with PNG Info > Send to txt2img 5 hit Generate, nothing is happening. For 3sec the progress bar is showing "Waiting" then it disappears, and there's Generate button again. But now it's completely broken, unable to generate anymore. Even when I try the Generate button in img2img, no shot neither.
What should have happened?
Well, it should do this: https://www.youtube.com/watch?v=JuKX7MvQYcc
But instead, it just does not work.
Simple.
Commit where the problem happens
webui: controlnet:
What browsers do you use to access the UI ?
Microsoft Edge
Command Line Arguments
--xformers
Console logs
D:\Stable WEBUI\stable-diffusion-webui>set PYTHON=
D:\Stable WEBUI\stable-diffusion-webui>set GIT=
D:\Stable WEBUI\stable-diffusion-webui>set VENV_DIR=
D:\Stable WEBUI\stable-diffusion-webui>set COMMANDLINE_ARGS= --xformers
D:\Stable WEBUI\stable-diffusion-webui>call webui.bat
venv "D:\Stable WEBUI\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Commit hash: 22bcc7be428c94e9408f589966c2040187245d81
Installing requirements for Web UI
Installing requirements for scikit_learn
Launching Web UI with arguments: --xformers
Opening SD in default browser
Civitai Helper: Get Custom Model Folder
Civitai Helper: Load setting from: D:\Stable WEBUI\stable-diffusion-webui\extensions\Stable-Diffusion-Webui-Civitai-Helper\setting.json
Loading weights [190ec8ef68] from D:\Stable WEBUI\stable-diffusion-webui\models\Stable-diffusion\jimEIDOMODE_version10.safetensors
Creating model from config: D:\Stable WEBUI\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading VAE weights specified in settings: D:\Stable WEBUI\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(43): 4ndreabotez, catgirlv1, corneo_bound_missionary, corneo_covering_breasts_one_arm, corneo_covering_breasts_two _hands, corneo_cowgirl, corneo_runny_makeup, corneo_side_doggy, corneo_spitroast, corneo_x_pasties, emb-rrf2, eonn, facev2, ghst-3000, gunslinger, gun slinger_2, H0GLEGACYv2, KingLich , lol_splash_base, microkini, micromini, miss_denight, mverse , nightborne, oilskin, pureerosface_v1, rz-neg-15-foran alog, Silvana, ulzzang-6500-v1.1, was-mecha, was-steampunk, yor_briar, zero_two_001, zero_two_007, bad-hands-5, bad-image-v2-39000, bad-picture-chill- 75v, bad_prompt_version2, By bad artist -neg, By bad artist -neg_2, easynegative, ng_deepnegative_v1_75t, Unspeakable-Horrors-Composition-4v
Textual inversion embeddings skipped(8): HorrorByDave, Lavastyle, rz-neg-foranalogportrait, rz-neg-general, xgiger, nartfixer, nfixer, nrealfixer
Model loaded in 3.4s (load weights from disk: 0.2s, create model: 0.5s, apply weights to model: 0.6s, apply half(): 0.5s, load VAE: 0.1s, move model t o device: 0.6s, load textual inversion embeddings: 0.9s).
D:\Stable WEBUI\stable-diffusion-webui\venv\lib\site-packages\gradio\deprecation.py:43: UserWarning: You have unused kwarg parameters in Row, please r emove them: {'equal_height': True}
warnings.warn(
D:\Stable WEBUI\stable-diffusion-webui\venv\lib\site-packages\gradio\deprecation.py:43: UserWarning: You have unused kwarg parameters in Tab, please r emove them: {'open': False}
warnings.warn(
[VRAMEstimator] Loaded benchmark data.
[Discord Rich Presence] Running Discord Rich Presence Extension, version 1.1.0
[Discord Rich Presence] Bug reporting -> https://github.com/davehornik/sd-discord-rich_presence
[Discord Rich Presence] 'pypresence' module is installed - skipping.
[Discord Rich Presence] Make sure that Game Activity is enabled in Discord.
[Discord Rich Presence] Should be running already if there's no error.
D:\Stable WEBUI\stable-diffusion-webui\venv\lib\site-packages\gradio\deprecation.py:43: UserWarning: You have unused kwarg parameters in Row, please r emove them: {'scale': 0.5}
warnings.warn(
Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
Startup time: 13.7s (import torch: 1.6s, import gradio: 1.1s, import ldm: 0.6s, other imports: 0.8s, list extensions: 0.2s, setup codeformer: 0.2s, lo ad scripts: 1.8s, load SD checkpoint: 3.5s, create ui: 3.8s, gradio launch: 0.1s).
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:03<00:00, 9.10it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 30/30 [00:01<00:00, 15.56it/s]
100%|████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 8.48it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████████| 8/8 [00:01<00:00, 7.66it/s]
T
Additional information
Good luck with the console logs since there's no info.
Which commit are you using for controlnet?
Latest, updated thru WebUI's "Check for Update"
I was trying one by one enabling them, loaded ControlNet in the last step, and then it started. Then I just deleted. I could try to downlaoad it again, and run it solo.
I tried to reproduce this issue using the multi-cn batch branch but I couldn't. I'll try in a clean env once I'm done with batch, if no one else looked into this yet.
I could try to downlaoad it again, and run it solo.
If you have time to do that, we could verify that it was not just a configuration problem from controlnet. If you could try to reproduce in a clean webui where only controlnet is installed, that could help narrow it down even more.
I tried to reproduce this issue using the multi-cn batch branch but I couldn't. I'll try in a clean env once I'm done with batch, if no one else looked into this yet.
I could try to downlaoad it again, and run it solo.
If you have time to do that, we could verify that it was not just a configuration problem from controlnet. If you could try to reproduce in a clean webui where only controlnet is installed, that could help narrow it down even more.
I just did that, I was just about to reply. And I can confirm it is not happening, while it is solo.
But the werid stuff is, everything was okay yesterday. I checked the latest updated plugins, and from some reason 5 of them got updated today. (but they were not in last month kinda) so Idk what happened, that forced these updates...
edit: the only thing I can do now, is leaving ControlNet on(and not enabling it as last) from the beginning, and see what is causing that. What a pain to try reproducing that error, with my 20 or so extensions...
So after a break from all of this, I managed to go one-by-one, and reproducing the steps on each one.
And as it seems to me, there is something going on between THIS and ControlNet. But since it was fine before ControlNet was updated in last 4 or so days, it is not fault of the webui-state extension.
There's log from console, but nothing sus in there...
D:\Stable WEBUI\stable-diffusion-webui>set PYTHON=
D:\Stable WEBUI\stable-diffusion-webui>set GIT=
D:\Stable WEBUI\stable-diffusion-webui>set VENV_DIR=
D:\Stable WEBUI\stable-diffusion-webui>set COMMANDLINE_ARGS= --xformers
D:\Stable WEBUI\stable-diffusion-webui>call webui.bat
venv "D:\Stable WEBUI\stable-diffusion-webui\venv\Scripts\Python.exe"
Python 3.10.11 (tags/v3.10.11:7d4cc5a, Apr 5 2023, 00:38:17) [MSC v.1929 64 bit (AMD64)]
Commit hash: 22bcc7be428c94e9408f589966c2040187245d81
Installing requirements for Web UI
Launching Web UI with arguments: --xformers
Opening SD in default browser
Loading weights [190ec8ef68] from D:\Stable WEBUI\stable-diffusion-webui\models\Stable-diffusion\jimEIDOMODE_version10.safetensors
Creating model from config: D:\Stable WEBUI\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Loading VAE weights specified in settings: D:\Stable WEBUI\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(43): 4ndreabotez, catgirlv1, corneo_bound_missionary, corneo_covering_breasts_one_arm, corneo_covering_breasts_two _hands, corneo_cowgirl, corneo_runny_makeup, corneo_side_doggy, corneo_spitroast, corneo_x_pasties, emb-rrf2, eonn, facev2, ghst-3000, gunslinger, gun slinger_2, H0GLEGACYv2, KingLich , lol_splash_base, microkini, micromini, miss_denight, mverse , nightborne, oilskin, pureerosface_v1, rz-neg-15-foran alog, Silvana, ulzzang-6500-v1.1, was-mecha, was-steampunk, yor_briar, zero_two_001, zero_two_007, bad-hands-5, bad-image-v2-39000, bad-picture-chill- 75v, bad_prompt_version2, By bad artist -neg, By bad artist -neg_2, easynegative, ng_deepnegative_v1_75t, Unspeakable-Horrors-Composition-4v
Textual inversion embeddings skipped(8): HorrorByDave, Lavastyle, rz-neg-foranalogportrait, rz-neg-general, xgiger, nartfixer, nfixer, nrealfixer
Model loaded in 2.6s (load weights from disk: 0.2s, create model: 0.4s, apply weights to model: 0.4s, apply half(): 0.3s, move model to device: 0.5s, load textual inversion embeddings: 0.8s).
D:\Stable WEBUI\stable-diffusion-webui\venv\lib\site-packages\gradio\deprecation.py:43: UserWarning: You have unused kwarg parameters in Tab, please r emove them: {'open': False}
warnings.warn(
Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
Startup time: 9.9s (import torch: 1.2s, import gradio: 0.8s, import ldm: 0.4s, other imports: 0.7s, list extensions: 0.2s, setup codeformer: 0.2s, loa d scripts: 1.1s, load SD checkpoint: 2.7s, create ui: 2.5s, gradio launch: 0.2s).
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:02<00:00, 11.13it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:01<00:00, 18.88it/s]
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 9.24it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 8.24it/s]
D:\Stable WEBUI\stable-diffusion-webui\venv\lib\site-packages\torch\functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ..\aten\src\ATen\native\TensorShape.cpp:3191.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
SwinIR tiles: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:02<00:00, 1.82it/s]
SD upscaling will process a total of 1 images tiled as 1x1 per upscale in a total of 1 batches.
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 9.52it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 8.45it/s]
Total progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 8/8 [00:00<00:00, 10.22it/s]
It's possible that state depends on something in controlnet that changed since there was an update to controlnet recently. Strangely enough, since I have not updated sd-webui-state on my side, I am unable to reproduce this issue. So maybe it is a bug on their side?
For now you may be able to just disable the controlnet state as a temporary workaround? Unless it is already disabled.
I did also found out about this, lmao.. https://github.com/AlUlkesh/stable-diffusion-webui-images-browser/issues/131
Its like a spiderweb of issues now :D Yea, I did turned off ControlNet since it was also messing up with the Image Browser as you could see in the link provided. Very strange. And also I made sure I did clean install all of them - same.
I had this same problem, just randomly started happening when using controlnet today for some reason. I had the image browser and the save state enabled extensions enabled.
after reading all this I simply unchecked image browser in my extensions tab and restarted the UI and now it works again :) I think that was the issue.....save state seems to be playing nicely with controlnet now.
Dang man I have spent hours of my time trying to get sd controlnet working with the webui and non of this have helped. I have reinstalled the main sd webui multiple time and same with the sd controlnet. It works for one image then freezes once generation is at 100%? Whyyyy
