sd-webui-controlnet icon indicating copy to clipboard operation
sd-webui-controlnet copied to clipboard

Published 20 hours ago • verified publisher icon Mikubill

[Bug]: sd-webui-controlnet doesn't work 【It can't work normally under linux, it has been tested in colab】

Open triggered96 opened this issue 2 years ago • 9 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

I've tried sd-webui-controlnet really hard, but it doesn't work. After nearly a week of research, I still can't recognize the skeleton, outline, etc. The result of the Preview annotator is either all white or all black.

I'm currently on the verge of a nervous breakdown. hope based on help

Steps to reproduce the problem

1 Note: I accidentally turned off the Enable above when I took a screenshot, but it was actually turned on

2

3

4

What should have happened?

I want to get a line drawing, or a human skeleton. Instead of being completely blank, or completely black. as shown in the picture above

Commit where the problem happens

webui: controlnet:

I don't seem to know how to do this

What browsers do you use to access the UI ?

No response

Command Line Arguments

I don't seem to know how to do this

Console logs

Loading model: control_scribble-fp16 [c508311e]
Loaded state_dict from [/home/ubuntu/1/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_scribble-fp16.safetensors]
ControlNet model control_scribble-fp16 [c508311e] loaded.
using mask as input
Loading preprocessor: scribble
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:49<00:00,  2.49s/it]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:49<00:00,  2.49s/it]
Closing server running on port: 7860




### Additional information

_No response_

triggered96 avatar Apr 06 '23 17:04 triggered96

By the way, I am running under ubuntu 20.4 LTS

COMMANDLINE_ARGS="--listen --xformers --enable-insecure-extension-access --theme dark" ./webui.sh

triggered96 avatar Apr 06 '23 17:04 triggered96

For the first image, the preprocessor is doing exactly what you told it do do. You do not need a preprocessor when passing a scribble directly, you need a preprocessor to derive a scribble from a real picture.

ljleb avatar Apr 06 '23 20:04 ljleb

Sorry I am a bit confused because there seems to be 2 different issues at play here.

For the first image, the preprocessor is doing exactly what you told it do do. You do not need a preprocessor when passing a scribble directly, you need a preprocessor to derive a scribble from a real picture.

For the second image, you may be encountering a problem with the preprocessor. Any logs when using the openpose preprocessor? (IIUC only the logs for scribbles have been included above)

ljleb avatar Apr 06 '23 20:04 ljleb

Sorry I am a bit confused because there seems to be 2 different issues at play here.

For the first image, the preprocessor is doing exactly what you told it do do. You do not need a preprocessor when passing a scribble directly, you need a preprocessor to derive a scribble from a real picture.

For the second image, you may be encountering a problem with the preprocessor. Any logs when using the openpose preprocessor? (IIUC only the logs for scribbles have been included above)

Loading weights [fc2511737a] from /home/ubuntu/1/stable-diffusion-webui/models/Stable-diffusion/chilloutmix_NiPrunedFp32Fix.safetensors
Creating model from config: /home/ubuntu/1/stable-diffusion-webui/configs/v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(1): EasyNegative
Model loaded in 7.2s (create model: 0.9s, apply weights to model: 2.2s, apply half(): 1.9s, load VAE: 1.5s, move model to device: 0.6s).
*Deforum ControlNet support: enabled*
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Loading model: control_canny-fp16 [e3fe7712]
Loaded state_dict from [/home/ubuntu/1/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_canny-fp16.safetensors]
ControlNet model control_canny-fp16 [e3fe7712] loaded.
using mask as input
Loading preprocessor: canny
  0%|                                                                                                                                                                                                        | 0/20 [00:00<?, ?it/s]Warning: StyleAdapter and cfg/guess mode may not works due to non-batch-cond inference
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:36<00:00,  1.80s/it]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:34<00:00,  1.75s/it]

*Deforum ControlNet support: enabled*
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Loading model: control_canny-fp16 [e3fe7712]
Loaded state_dict from [/home/ubuntu/1/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_canny-fp16.safetensors]
ControlNet model control_canny-fp16 [e3fe7712] loaded.
using mask as input
Loading preprocessor: canny
  0%|                                                                                                                                                                                                        | 0/20 [00:00<?, ?it/s]Warning: StyleAdapter and cfg/guess mode may not works due to non-batch-cond inference
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:36<00:00,  1.80s/it]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:34<00:00,  1.75s/it]
Loading model: control_openpose-fp16 [9ca67cc5]
Loaded state_dict from [/home/ubuntu/1/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_openpose-fp16.safetensors]
ControlNet model control_openpose-fp16 [9ca67cc5] loaded.
using mask as input
Loading preprocessor: openpose
  0%|                                                                                                                                                                                                        | 0/20 [00:00<?, ?it/s]Warning: StyleAdapter and cfg/guess mode may not works due to non-batch-cond inference
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:35<00:00,  1.76s/it]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:34<00:00,  1.74s/it]

Loading model from cache: control_openpose-fp16 [9ca67cc5]
using mask as input
Loading preprocessor: openpose
  0%|                                                                                                                                                                                                        | 0/20 [00:00<?, ?it/s]Warning: StyleAdapter and cfg/guess mode may not works due to non-batch-cond inference
 60%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌                                                                            | 12/20 [00:21<00:14,  1.76s/it]
Total progress:  60%|█████████████████████████████████████████████████████████████████████████████████████████████████████████                                                                      | 12/20 [00:19<00:13,  1.74s/it]

It seems to be stuck at Loading preprocessor: xxx 0%|

Does this mean it's not normal? How to solve it? I have a big head now

triggered96 avatar Apr 07 '23 02:04 triggered96

1 2 3

Launching Web UI with arguments: --listen --disable-safe-unpickle --no-half-vae --xformers --enable-insecure-extension-access --medvram --theme dark
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
Loading weights [fc2511737a] from /home/ubuntu/1/stable-diffusion-webui/models/Stable-diffusion/chilloutmix_NiPrunedFp32Fix.safetensors
Creating model from config: /home/ubuntu/1/stable-diffusion-webui/configs/v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying cross attention optimization (Doggettx).
Textual inversion embeddings loaded(1): EasyNegative
Model loaded in 5.8s (create model: 0.9s, apply weights to model: 1.8s, apply half(): 1.5s, load VAE: 1.5s).
*Deforum ControlNet support: enabled*
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:26<00:00,  1.34s/it]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:27<00:00,  1.36s/it]
Loading model: control_openpose-fp16 [9ca67cc5]
Loaded state_dict from [/home/ubuntu/1/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_openpose-fp16.safetensors]
ControlNet model control_openpose-fp16 [9ca67cc5] loaded.
using mask as input
Loading preprocessor: openpose
  0%|                                                                                                                                                                                                        | 0/20 [00:00<?, ?it/s]Warning: StyleAdapter and cfg/guess mode may not works due to non-batch-cond inference
 40%|████████████████████████████████████████████████████████████████████████████▊                                                                                                                   | 8/20 [00:14<00:22,  1.84s/it]
Total progress:  40%|██████████████████████████████████████████████████████████████████████▍                                                                                                         | 8/20 [00:15<00:22,  1.91s/it]
Loading model from cache: control_openpose-fp16 [9ca67cc5]
using mask as input
Loading preprocessor: openpose
  0%|                                                                                                                                                                                                        | 0/20 [00:00<?, ?it/s]Warning: StyleAdapter and cfg/guess mode may not works due to non-batch-cond inference
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:35<00:00,  1.78s/it]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:35<00:00,  1.79s/it]
Closing server running on port: 7860
Restarting UI...
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Loading model: control_openpose-fp16 [9ca67cc5]
Loaded state_dict from [/home/ubuntu/1/stable-diffusion-webui/extensions/sd-webui-controlnet/models/control_openpose-fp16.safetensors]
ControlNet model control_openpose-fp16 [9ca67cc5] loaded.
using mask as input
Loading preprocessor: openpose
  0%|                                                                                                                                                                                                        | 0/20 [00:00<?, ?it/s]Warning: StyleAdapter and cfg/guess mode may not works due to non-batch-cond inference
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [02:00<00:00,  6.03s/it]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [02:00<00:00,  6.03s/it]
Loading model from cache: control_openpose-fp16 [9ca67cc5]
using mask as input
Loading preprocessor: openpose
  0%|                                                                                                                                                                                                        | 0/20 [00:00<?, ?it/s]Warning: StyleAdapter and cfg/guess mode may not works due to non-batch-cond inference
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [01:10<00:00,  3.51s/it]
Total progress: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [01:10<00:00,  3.53s/it]

triggered96 avatar Apr 07 '23 03:04 triggered96

I still can't get it to work, my mental state is really bad right now.

triggered96 avatar Apr 07 '23 03:04 triggered96

I tried colab, still the same, pure black. https://colab.research.google.com/github/camenduru/stable-diffusion-webui-colab/blob/main/nightly/chillout_mix_webui_colab.ipynb

https://github.com/camenduru/stable-diffusion-webui-colab

triggered96 avatar Apr 07 '23 11:04 triggered96

I wonder if it is under linux that the pictures uploaded by pictures cannot be identified? I am in colab, and the camenduru/stable-diffusion-webui-colab project is still very popular, and I found that the same problem exists. You can reproduce this problem in colab.

triggered96 avatar Apr 07 '23 11:04 triggered96

I think it should be a bug, because I see that many Windows users have not encountered this problem, but this problem will occur under Linux.

triggered96 avatar Apr 07 '23 11:04 triggered96