sd-webui-controlnet icon indicating copy to clipboard operation
sd-webui-controlnet copied to clipboard

Published 20 hours ago • verified publisher icon Mikubill

[Bug]: reference image + batch size 4 = different results than batch size 1

Open dtxn opened this issue 9 months ago • 3 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

When I use reference_adain with default batch size 1 and repeat image generation with the same seed I get always the same result as it suppose to be. When I use reference_adain with default batch size 4 and then select the seed I liked from those 4 generated images and try to repeat generation, I get different results. They are not drastically different, but esthetically often not good (different shape mountain, instead of normal trees only sticks and so on). P.S. If to repeat the same without Controlnet enabled, then everything works as suppose to be.

Steps to reproduce the problem

  1. Enable controlnet with reference_adain, add random picture as reference image
  2. Select batch size of 4 in Auto1111
  3. Generate 4 images
  4. Select 1 image you like from those 4 and repeat the same seed of that liked image
  5. Generate
  6. Get different picture than it was rendered in batch 4

What should have happened?

Suppose to be the same result in batch 1 with the same seed as in batch 4 generated images.

Commit where the problem happens

webui: Auto1111 controlnet: v1.1.410

What browsers do you use to access the UI ?

No response

Command Line Arguments

No

List of enabled extensions

ControlNet v1.1.410

Console logs

Initializing Dreambooth
Dreambooth revision: 1a1d1621086a4725fda1200256f319c845dc7a8a
Successfully installed accelerate-0.23.0 fastapi-0.94.1 transformers-4.32.1

[!] xformers version 0.0.20 installed.
[+] torch version 2.0.1+cu118 installed.
[+] torchvision version 0.15.2+cu118 installed.
[+] accelerate version 0.23.0 installed.
[+] diffusers version 0.21.4 installed.
[+] transformers version 4.32.1 installed.
[+] bitsandbytes version 0.41.1 installed.
Installing pyqt5 requirement for depthmap script
Launching Web UI with arguments: --xformers
[-] ADetailer initialized. version: 23.10.0, num models: 9
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
[AddNet] Updating model hashes...
0it [00:00, ?it/s]
2023-10-19 22:11:44,878 - ControlNet - INFO - ControlNet v1.1.410
ControlNet preprocessor location: C:\Users\drais\stable-diffusion-webui\extensions\sd-webui-controlnet\annotator\downloads
2023-10-19 22:11:45,181 - ControlNet - INFO - ControlNet v1.1.410
Loading weights [ec41bd2a82] from C:\Users\drais\stable-diffusion-webui\models\Stable-diffusion\photon_v1.safetensors
*Deforum ControlNet support: enabled*
Creating model from config: C:\Users\drais\stable-diffusion-webui\configs\v1-inference.yaml
False

===================================BUG REPORT===================================
================================================================================
The following directories listed in your path were found to be non-existent: {WindowsPath('tmp/restart')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
DEBUG: Possible options found for libcudart.so: set()
CUDA SETUP: PyTorch settings found: CUDA_VERSION=118, Highest Compute Capability: 8.6.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Loading binary C:\Users\drais\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA runtime library was not detected.
CUDA SETUP: Solution 1: To solve the issue the libcudart.so location needs to be added to the LD_LIBRARY_PATH variable
CUDA SETUP: Solution 1a): Find the cuda runtime library via: find / -name libcudart.so 2>/dev/null
CUDA SETUP: Solution 1b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_1a
CUDA SETUP: Solution 1c): For a permanent solution add the export from 1b into your .bashrc file, located at ~/.bashrc
CUDA SETUP: Solution 2: If no library was found in step 1a) you need to install CUDA.
CUDA SETUP: Solution 2a): Download CUDA install script: wget https://github.com/TimDettmers/bitsandbytes/blob/main/cuda_install.sh
CUDA SETUP: Solution 2b): Install desired CUDA version to desired location. The syntax is bash cuda_install.sh CUDA_VERSION PATH_TO_INSTALL_INTO.
CUDA SETUP: Solution 2b): For example, "bash cuda_install.sh 113 ~/local/" will download CUDA 11.3 and install into the folder ~/local

===================================BUG REPORT===================================
================================================================================
The following directories listed in your path were found to be non-existent: {WindowsPath('tmp/restart')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
DEBUG: Possible options found for libcudart.so: set()
CUDA SETUP: PyTorch settings found: CUDA_VERSION=118, Highest Compute Capability: 8.6.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Loading binary C:\Users\drais\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA runtime library was not detected.
CUDA SETUP: Solution 1: To solve the issue the libcudart.so location needs to be added to the LD_LIBRARY_PATH variable
CUDA SETUP: Solution 1a): Find the cuda runtime library via: find / -name libcudart.so 2>/dev/null
CUDA SETUP: Solution 1b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_1a
CUDA SETUP: Solution 1c): For a permanent solution add the export from 1b into your .bashrc file, located at ~/.bashrc
CUDA SETUP: Solution 2: If no library was found in step 1a) you need to install CUDA.
CUDA SETUP: Solution 2a): Download CUDA install script: wget https://github.com/TimDettmers/bitsandbytes/blob/main/cuda_install.sh
CUDA SETUP: Solution 2b): Install desired CUDA version to desired location. The syntax is bash cuda_install.sh CUDA_VERSION PATH_TO_INSTALL_INTO.
CUDA SETUP: Solution 2b): For example, "bash cuda_install.sh 113 ~/local/" will download CUDA 11.3 and install into the folder ~/local

===================================BUG REPORT===================================
================================================================================
The following directories listed in your path were found to be non-existent: {WindowsPath('tmp/restart')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
DEBUG: Possible options found for libcudart.so: set()
CUDA SETUP: PyTorch settings found: CUDA_VERSION=118, Highest Compute Capability: 8.6.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Loading binary C:\Users\drais\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA runtime library was not detected.
CUDA SETUP: Solution 1: To solve the issue the libcudart.so location needs to be added to the LD_LIBRARY_PATH variable
CUDA SETUP: Solution 1a): Find the cuda runtime library via: find / -name libcudart.so 2>/dev/null
CUDA SETUP: Solution 1b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_1a
CUDA SETUP: Solution 1c): For a permanent solution add the export from 1b into your .bashrc file, located at ~/.bashrc
CUDA SETUP: Solution 2: If no library was found in step 1a) you need to install CUDA.
CUDA SETUP: Solution 2a): Download CUDA install script: wget https://github.com/TimDettmers/bitsandbytes/blob/main/cuda_install.sh
CUDA SETUP: Solution 2b): Install desired CUDA version to desired location. The syntax is bash cuda_install.sh CUDA_VERSION PATH_TO_INSTALL_INTO.
CUDA SETUP: Solution 2b): For example, "bash cuda_install.sh 113 ~/local/" will download CUDA 11.3 and install into the folder ~/local

===================================BUG REPORT===================================
================================================================================
The following directories listed in your path were found to be non-existent: {WindowsPath('tmp/restart')}
CUDA_SETUP: WARNING! libcudart.so not found in any environmental path. Searching in backup paths...
The following directories listed in your path were found to be non-existent: {WindowsPath('/usr/local/cuda/lib64')}
DEBUG: Possible options found for libcudart.so: set()
CUDA SETUP: PyTorch settings found: CUDA_VERSION=118, Highest Compute Capability: 8.6.
CUDA SETUP: To manually override the PyTorch CUDA version please see:https://github.com/TimDettmers/bitsandbytes/blob/main/how_to_use_nonpytorch_cuda.md
CUDA SETUP: Loading binary C:\Users\drais\stable-diffusion-webui\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.so...
argument of type 'WindowsPath' is not iterable
CUDA SETUP: Problem: The main issue seems to be that the main CUDA runtime library was not detected.
CUDA SETUP: Solution 1: To solve the issue the libcudart.so location needs to be added to the LD_LIBRARY_PATH variable
CUDA SETUP: Solution 1a): Find the cuda runtime library via: find / -name libcudart.so 2>/dev/null
CUDA SETUP: Solution 1b): Once the library is found add it to the LD_LIBRARY_PATH: export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:FOUND_PATH_FROM_1a
CUDA SETUP: Solution 1c): For a permanent solution add the export from 1b into your .bashrc file, located at ~/.bashrc
CUDA SETUP: Solution 2: If no library was found in step 1a) you need to install CUDA.
CUDA SETUP: Solution 2a): Download CUDA install script: wget https://github.com/TimDettmers/bitsandbytes/blob/main/cuda_install.sh
CUDA SETUP: Solution 2b): Install desired CUDA version to desired location. The syntax is bash cuda_install.sh CUDA_VERSION PATH_TO_INSTALL_INTO.
CUDA SETUP: Solution 2b): For example, "bash cuda_install.sh 113 ~/local/" will download CUDA 11.3 and install into the folder ~/local
Loading VAE weights specified in settings: C:\Users\drais\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 108.2s (prepare environment: 65.1s, import torch: 6.1s, import gradio: 1.4s, setup paths: 3.8s, initialize shared: 0.4s, other imports: 3.6s, setup codeformer: 0.6s, load scripts: 11.6s, create ui: 11.1s, gradio launch: 4.1s).Applying attention optimization: xformers... done.
Model loaded in 18.9s (load weights from disk: 2.0s, create model: 1.2s, apply weights to model: 7.9s, apply half(): 0.9s, load VAE: 4.3s, calculate empty prompt: 2.5s).
Restoring base VAE
Applying attention optimization: xformers... done.
VAE weights loaded.
Loading VAE weights specified in settings: C:\Users\drais\stable-diffusion-webui\models\VAE\vae-ft-mse-840000-ema-pruned.safetensors
Applying attention optimization: xformers... done.
VAE weights loaded.
2023-10-19 22:17:15,671 - ControlNet - INFO - Loading preprocessor: reference_adain
2023-10-19 22:17:15,672 - ControlNet - INFO - preprocessor resolution = 640
2023-10-19 22:17:16,243 - ControlNet - INFO - ControlNet Hooked - Time = 0.9826855659484863
  0%|                                                                                           | 0/30 [00:00<?, ?it/s]2023-10-19 22:17:18,591 - ControlNet - INFO - ControlNet used torch.float16 VAE to encode torch.Size([1, 4, 80, 192]).
100%|██████████████████████████████████████████████████████████████████████████████████| 30/30 [00:18<00:00,  1.59it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 30/30 [00:17<00:00,  1.76it/s]
2023-10-19 22:18:29,864 - ControlNet - INFO - Loading preprocessor: reference_adain████| 30/30 [00:17<00:00,  1.82it/s]
2023-10-19 22:18:29,876 - ControlNet - INFO - preprocessor resolution = 640
2023-10-19 22:18:30,371 - ControlNet - INFO - ControlNet Hooked - Time = 0.5238332748413086

Additional information

None

dtxn avatar Oct 19 '23 21:10 dtxn