AUTOMATIC1111
[Bug]: Weird class images generated from dreambooth
Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
What happened?
I wrote in "full body cat" under Concepts->Prompts->Class prompt and asked it to generate class images but they're coming out horribly. My model uses the checkpoint "v1-5-pruned-emaonly.ckpt".
What's weird is that when I just use text2img with the same checkpoint I get great results. But as soon as I have it create class images then I get these weird pixelated outputs. This didn't use to happen before. Not sure if it's something wrong with dreambooth or how I filled out the UI as it changed since I last used it. Thanks for the help!
screenshots of my configurations:
the monstrosities:
Steps to reproduce the problem
Go to dreambooth, Go to the concepts tab, Under prompts->class prompt write in "full body cat". Under image generation -> Class Images Per Instance Image write in 1. Under directories->Dataset Directory point to a folder with at least 1 picture
Then under the generate tap click generate class images and you should see the output
What should have happened?
A decent image of a cat should have been produced, just like it can with text2img
Commit where the problem happens
python: 3.10.0 • torch: 1.13.1+cu117 • xformers: 0.0.16rc425 • gradio: 3.16.2 • commit: 226d840e • checkpoint: e1441589a6
What platforms do you use to access the UI ?
Windows
What browsers do you use to access the UI ?
Google Chrome
Command Line Arguments
--xformers --medvram --opt-split-attention --autolaunch
List of extensions
depthmap2mask sd_dreambooth_extension stable-diffusion-webui-depthmap-script stable-diffusion-webui-instruct-pix2pix stable-diffusion-webui-pixelization
Console logs
#######################################################################################################
Initializing Dreambooth
If submitting an issue on github, please provide the below text for debugging purposes:
Python revision: 3.10.0 (tags/v3.10.0:b494f59, Oct 4 2021, 19:00:18) [MSC v.1929 64 bit (AMD64)]
Dreambooth revision: b5b67040502bab07225cae02244fa9b77d94f160
SD-WebUI revision: 226d840e84c5f306350b0681945989b86760e616
Checking Dreambooth requirements...
[+] bitsandbytes version 0.35.0 installed.
[+] diffusers version 0.10.2 installed.
[+] transformers version 4.25.1 installed.
[+] xformers version 0.0.16rc425 installed.
[+] torch version 1.13.1+cu117 installed.
[+] torchvision version 0.14.1+cu117 installed.
#######################################################################################################
Installing imageio-ffmpeg requirement for depthmap script
Installing pyqt5 requirement for depthmap script
Launching Web UI with arguments: --xformers --medvram --opt-split-attention --autolaunch
SD-Webui API layer loaded
Loading weights [e1441589a6] from C:\Users\alexy\Desktop\ai_stuff\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned.ckpt
Creating model from config: C:\Users\alexy\Desktop\ai_stuff\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying xformers cross attention optimization.
Textual inversion embeddings loaded(0):
Model loaded in 78.3s (load weights from disk: 68.4s, load config: 1.0s, create model: 2.9s, apply weights to model: 3.4s, apply half(): 1.9s, load VAE: 0.3s, load textual inversion embeddings: 0.3s).
Running on local URL: http://127.0.0.1:7860
To create a public link, set `share=True` in `launch()`.
Generating class images...
Pre-processing fat_cats: 100%|████████████████████████████████████████████████████████| 35/35 [00:00<00:00, 699.98it/s]
Pre-processing classifiers_0: 100%|██████████████████████████████████████████████████████| 5/5 [00:00<00:00, 38.42it/s]
Concept requires 10 class images per instance image.
Sorting instance images: 100%|██████████████████████████████████████████████████████| 13/13 [00:00<00:00, 13053.85it/s]
We need a total of 345 class images.
Generating 345 class images for training...
Generating class images 0/345:: 0%| | 0/345 [00:00<?, ?it/s]Loading model: v1-5-pruned-emaonly
Loading weights [cc6cb27103] from C:\Users\alexy\Desktop\ai_stuff\stable-diffusion-webui\models\Stable-diffusion\v1-5-pruned-emaonly.ckpt
Creating model from config: C:\Users\alexy\Desktop\ai_stuff\stable-diffusion-webui\configs\v1-inference.yaml
LatentDiffusion: Running in eps-prediction mode
DiffusionWrapper has 859.52 M params.
Applying xformers cross attention optimization.
Model loaded in 45.7s (load weights from disk: 41.6s, create model: 0.6s, apply weights to model: 0.9s, apply half(): 0.9s, load VAE: 1.5s).
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:23<00:00, 1.16s/it]
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:08<00:00, 2.40it/s]
Generating class images 2/345:: 0%|▏ | 1/345 [00:00<?, ?it/s]Generated 2 new class images.
Generating class images 2/345:: 0%|▏ | 1/345 [00:00<01:07, 5.13it/s]
Restored system models.
Additional information
No response
this should be reported in dreambooth repository (its a separate extension), not webui
https://github.com/d8ahazard/sd_dreambooth_extension/issues/883
Similar, comes from Xformers
