sd_dreambooth_extension icon indicating copy to clipboard operation
sd_dreambooth_extension copied to clipboard

Published 20 hours ago • verified publisher icon d8ahazard

Lora appears to be created during training but once complete it doesn't produce expected results[Bug]:

Open klubkelli opened this issue 1 year ago • 12 comments

Is there an existing issue for this?

  • [X] I have searched the existing issues and checked the recent builds/commits of both this extension and the webui

What happened?

I trained a Lora using images of a person. During training, it seems like it's working on when it's done with the training. I go to use the image, and it doesn't produce images of the girl.

This is an early training image at about 2700 steps. Not great, but clearly, it is the expected result.

sample_2450-0

When the training is complete, and I got to use the lora it created, it doesn't work. The file appeared to have worked during training. But not when it saves.

Training finished. Total lifetime steps: 3500 sample_3500-0

I run the prompt "photo of zwx woman lora:ghiles_3500:1" and this is what it gives me

00003-626981190

I try some of the saved lora files that it made during training. Same issue.

Steps to reproduce the problem

Train lora. Use lora.

Commit and libraries

Installed version of pytorch_optimizer: 2.12.0 [Dreambooth] pytorch_optimizer v2.12.0 is already installed. Installed version of Pillow: 9.5.0 [Dreambooth] Pillow is already installed. Installed version of tqdm: 4.66.1 [Dreambooth] tqdm is already installed. Installed version of tomesd: 0.1.3 [Dreambooth] tomesd v0.1.2 is already installed. Installed version of tensorboard: 2.13.0 [Dreambooth] tensorboard v2.13.0 is already installed. [+] torch version 2.0.1+cu118 installed. [+] torchvision version 0.15.2+cu118 installed. [+] accelerate version 0.21.0 installed. [+] diffusers version 0.25.0 installed. [+] bitsandbytes version 0.41.2.post2 installed. [!] xformers NOT installed. Launching Web UI with arguments: bin C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll no module 'xformers'. Processing without... no module 'xformers'. Processing without... No module 'xformers'. Proceeding without it.

Command Line Arguments

No

Console logs

venv "C:\stable-diffusion-webui-master\venv\Scripts\Python.exe"
fatal: not a git repository (or any of the parent directories): .git
fatal: not a git repository (or any of the parent directories): .git
Python 3.10.6 (tags/v3.10.6:9c7b4bd, Aug  1 2022, 21:53:49) [MSC v.1932 64 bit (AMD64)]
Version: 1.7.0
Commit hash: <none>
Installing requirements
bin C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
If submitting an issue on github, please provide the full startup log for debugging purposes.

Initializing Dreambooth
Dreambooth revision: 71c3465b6c866050b147c58e2caf41984df2cf45
Checking xformers...
Checking bitsandbytes...
Checking bitsandbytes (Windows)
Virtual environment path: C:\stable-diffusion-webui-master\venv
Checking for C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda111.dll
Found windows BNB DLL C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda111.dll
Checking Dreambooth requirements...
Installed version of accelerate: 0.21.0
[Dreambooth] accelerate v0.21.0 is already installed.
Installed version of dadaptation: 3.2
[Dreambooth] dadaptation v3.2 is already installed.
Installed version of diffusers: 0.25.0
[Dreambooth] diffusers v0.25.0 is already installed.
Installed version of discord-webhook: 1.3.0
[Dreambooth] discord-webhook v1.3.0 is already installed.
Installed version of fastapi: 0.94.0
[Dreambooth] fastapi is already installed.
Installed version of gitpython: 3.1.32
[Dreambooth] gitpython v3.1.40 is not installed.
Successfully installed gitpython-3.1.41

Installed version of pytorch_optimizer: 2.12.0
[Dreambooth] pytorch_optimizer v2.12.0 is already installed.
Installed version of Pillow: 9.5.0
[Dreambooth] Pillow is already installed.
Installed version of tqdm: 4.66.1
[Dreambooth] tqdm is already installed.
Installed version of tomesd: 0.1.3
[Dreambooth] tomesd v0.1.2 is already installed.
Installed version of tensorboard: 2.13.0
[Dreambooth] tensorboard v2.13.0 is already installed.
[+] torch version 2.0.1+cu118 installed.
[+] torchvision version 0.15.2+cu118 installed.
[+] accelerate version 0.21.0 installed.
[+] diffusers version 0.25.0 installed.
[+] bitsandbytes version 0.41.2.post2 installed.
[!] xformers NOT installed.
Launching Web UI with arguments:
bin C:\stable-diffusion-webui-master\venv\lib\site-packages\bitsandbytes\libbitsandbytes_cuda118.dll
no module 'xformers'. Processing without...
no module 'xformers'. Processing without...
No module 'xformers'. Proceeding without it.
Style database not found: C:\stable-diffusion-webui-master\styles.csv
Loading weights [6ce0161689] from C:\stable-diffusion-webui-master\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors
Creating model from config: C:\stable-diffusion-webui-master\configs\v1-inference.yaml
Running on local URL:  http://127.0.0.1:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 16.5s (prepare environment: 10.1s, import torch: 2.3s, import gradio: 0.7s, setup paths: 0.8s, initialize shared: 0.2s, other imports: 0.3s, load scripts: 1.1s, create ui: 0.6s, gradio launch: 0.3s).
Advanced elements visible: False
Applying attention optimization: Doggettx... done.
Model loaded in 3.0s (load weights from disk: 0.4s, create model: 0.5s, apply weights to model: 1.8s, apply half(): 0.1s, calculate empty prompt: 0.1s).
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 10.33it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 12.69it/s]
Extracting config from C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\dreambooth\..\configs\v1-training-unfrozen.yaml
Extracting checkpoint from C:\stable-diffusion-webui-master\models\Stable-diffusion\v1-5-pruned-emaonly.safetensors
C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Duration: 00:00:10
Updating scheduler name to: DDIM
Advanced elements visible: True
Custom model name is ghiles
Initializing dreambooth training...
                                                                                                                       C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Init dataset!set:   0%|                                                                           | 0/1 [00:00<?, ?it/s]
Preparing Dataset (With Caching)
Bucket 0 (512, 512, 0) - Instance Images: 65 | Class Images: 0 | Max Examples/batch:  65
                                                                                                                       Saving cache!ed latents...: 100%|███████████████████████████████████████████████████████| 65/65 [00:03<00:00, 23.30it/s]
Total Buckets 1 - Instance Images: 65 | Class Images: 0 | Max Examples/batch: 65

Total images / batch: 65, total examples: 65███████████████████████████████████████████| 65/65 [00:03<00:00, 23.30it/s]
                  Initializing bucket counter!
Steps:   0%|                                                                                  | 0/3250 [00:00<?, ?it/s]C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\models\attention_processor.py:1946: FutureWarning: `LoRAAttnProcessor2_0` is deprecated and will be removed in version 0.26.0. Make sure use AttnProcessor2_0 instead by settingLoRA layers to `self.{to_q,to_k,to_v,to_out[0]}.lora_layer` respectively. This will be done automatically when using `LoraLoaderMixin.load_lora_weights`
  deprecate(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.28it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.61it/s]
Saving Lora Weights...:   0%|                                                                    | 0/1 [00:00<?, ?it/s]Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:14<00:28, 14.50s/it]
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  22%|████████▌                               | 700/3250 [01:42<04:46,  8.91it/s, loss=0.23, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.34it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.59it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.01s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  32%|████████████▎                         | 1050/3250 [02:40<04:08,  8.85it/s, loss=0.106, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.65it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.53it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.22s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  43%|████████████████▊                      | 1400/3250 [03:39<03:22,  9.12it/s, loss=0.24, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.27it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.48it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.28s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  54%|████████████████████▍                 | 1750/3250 [04:37<02:54,  8.62it/s, loss=0.123, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:01<00:00,  6.90it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.52it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.12s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  65%|████████████████████████▌             | 2100/3250 [05:35<02:11,  8.75it/s, loss=0.148, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.11it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.55it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.23s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  75%|███████████████████████████▉         | 2450/3250 [06:34<01:29,  8.89it/s, loss=0.0861, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:01<00:00,  6.90it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.50it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.42s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  86%|████████████████████████████████▋     | 2800/3250 [07:33<00:49,  9.04it/s, loss=0.168, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.57it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.61it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.11s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps:  97%|███████████████████████████████████▊ | 3150/3250 [08:30<00:10,  9.10it/s, loss=0.0699, lr=0.0001, vram=5.8]C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.48it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.56it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.35s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
Steps: : 3500it [09:28,  9.15it/s, loss=0.0461, lr=0.0001, vram=5.8]                                                   C:\stable-diffusion-webui-master\venv\lib\site-packages\transformers\models\clip\feature_extraction_clip.py:28: FutureWarning: The class CLIPFeatureExtractor is deprecated and will be removed in version 5 of Transformers. Please use CLIPImageProcessor instead.
  warnings.warn(
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:00<00:00,  7.55it/s]
Loading pipeline components...: 100%|████████████████████████████████████████████████████| 7/7 [00:02<00:00,  2.61it/s]
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:805: FutureWarning: `_modify_text_encoder` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_modify_text_encoder", "0.27", LORA_DEPRECATION_MESSAGE)
C:\stable-diffusion-webui-master\venv\lib\site-packages\diffusers\loaders\lora.py:777: FutureWarning: `_remove_text_encoder_monkey_patch_classmethod` is deprecated and will be removed in version 0.27. You are using an old version of LoRA backend. This will be deprecated in the next releases in favor of PEFT make sure to install the latest PEFT and transformers packages in the future.
  deprecate("_remove_text_encoder_monkey_patch_classmethod", "0.27", LORA_DEPRECATION_MESSAGE)
                                                                                                                       Model name: ghiless:  33%|█████████████████████▎                                          | 1/3 [00:13<00:26, 13.33s/it]
Log file updated, re-parsing: C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\dreambooth\events.out.tfevents.1706653508.DESKTOP-QGNRT35.1928.0
C:\stable-diffusion-webui-master\extensions\sd_dreambooth_extension\helpers\log_parser.py:324: FutureWarning: DataFrame.fillna with 'method' is deprecated and will raise in a future version. Use obj.ffill() or obj.bfill() instead.
  all_df_loss = all_df_loss.fillna(method="ffill")
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\loss_plot_0.png
Saving C:\stable-diffusion-webui-master\models\dreambooth\ghiles\logging\ram_plot_0.png
Cleanup log parse.
                                                                                                                       Cleanup completed.                                                                                | 0/3 [00:00<?, ?it/s]
Steps: : 3500it [09:48,  5.95it/s, loss=0.0461, lr=0.0001, vram=5.8]
  0%|                                                                                            | 0/3 [00:00<?, ?it/s]
Duration: 00:09:55
Duration: 00:09:56
loading network C:\stable-diffusion-webui-master\models\Lora\ghiles_3500.safetensors: AssertionError
Traceback (most recent call last):
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 279, in load_networks
    net = load_network(name, network_on_disk)
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 223, in load_network
    raise AssertionError(f"Could not find a module type (out of {', '.join([x.__class__.__name__ for x in module_types])}) that would accept those keys: {', '.join(weights.w)}")
AssertionError: Could not find a module type (out of ModuleTypeLora, ModuleTypeHada, ModuleTypeIa3, ModuleTypeLokr, ModuleTypeFull, ModuleTypeNorm, ModuleTypeGLora, ModuleTypeOFT) that would accept those keys: 0.attn1.to_k.lora_down.weight, 0.attn1.to_k.lora_up.weight, 0.attn1.to_out.0.lora_down.weight, 0.attn1.to_out.0.lora_up.weight, 0.attn1.to_q.lora_down.weight, 0.attn1.to_q.lora_up.weight, 0.attn1.to_v.lora_down.weight, 0.attn1.to_v.lora_up.weight, 0.attn2.to_k.lora_down.weight, 0.attn2.to_k.lora_up.weight, 0.attn2.to_out.0.lora_down.weight, 0.attn2.to_out.0.lora_up.weight, 0.attn2.to_q.lora_down.weight, 0.attn2.to_q.lora_up.weight, 0.attn2.to_v.lora_down.weight, 0.attn2.to_v.lora_up.weight

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 11.43it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:01<00:00, 12.31it/s]
loading network C:\stable-diffusion-webui-master\models\Lora\ghiles_3500.safetensors: AssertionError1<00:00, 12.51it/s]
Traceback (most recent call last):
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 279, in load_networks
    net = load_network(name, network_on_disk)
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 223, in load_network
    raise AssertionError(f"Could not find a module type (out of {', '.join([x.__class__.__name__ for x in module_types])}) that would accept those keys: {', '.join(weights.w)}")
AssertionError: Could not find a module type (out of ModuleTypeLora, ModuleTypeHada, ModuleTypeIa3, ModuleTypeLokr, ModuleTypeFull, ModuleTypeNorm, ModuleTypeGLora, ModuleTypeOFT) that would accept those keys: 0.attn1.to_k.lora_down.weight, 0.attn1.to_k.lora_up.weight, 0.attn1.to_out.0.lora_down.weight, 0.attn1.to_out.0.lora_up.weight, 0.attn1.to_q.lora_down.weight, 0.attn1.to_q.lora_up.weight, 0.attn1.to_v.lora_down.weight, 0.attn1.to_v.lora_up.weight, 0.attn2.to_k.lora_down.weight, 0.attn2.to_k.lora_up.weight, 0.attn2.to_out.0.lora_down.weight, 0.attn2.to_out.0.lora_up.weight, 0.attn2.to_q.lora_down.weight, 0.attn2.to_q.lora_up.weight, 0.attn2.to_v.lora_down.weight, 0.attn2.to_v.lora_up.weight

100%|██████████████████████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 10.50it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 10.71it/s]
loading network C:\stable-diffusion-webui-master\models\Lora\ghiles_3150.safetensors: AssertionError3<00:00, 10.71it/s]
Traceback (most recent call last):
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 279, in load_networks
    net = load_network(name, network_on_disk)
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 223, in load_network
    raise AssertionError(f"Could not find a module type (out of {', '.join([x.__class__.__name__ for x in module_types])}) that would accept those keys: {', '.join(weights.w)}")
AssertionError: Could not find a module type (out of ModuleTypeLora, ModuleTypeHada, ModuleTypeIa3, ModuleTypeLokr, ModuleTypeFull, ModuleTypeNorm, ModuleTypeGLora, ModuleTypeOFT) that would accept those keys: 0.attn1.to_k.lora_down.weight, 0.attn1.to_k.lora_up.weight, 0.attn1.to_out.0.lora_down.weight, 0.attn1.to_out.0.lora_up.weight, 0.attn1.to_q.lora_down.weight, 0.attn1.to_q.lora_up.weight, 0.attn1.to_v.lora_down.weight, 0.attn1.to_v.lora_up.weight, 0.attn2.to_k.lora_down.weight, 0.attn2.to_k.lora_up.weight, 0.attn2.to_out.0.lora_down.weight, 0.attn2.to_out.0.lora_up.weight, 0.attn2.to_q.lora_down.weight, 0.attn2.to_q.lora_up.weight, 0.attn2.to_v.lora_down.weight, 0.attn2.to_v.lora_up.weight

100%|██████████████████████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.28it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.02it/s]
loading network C:\stable-diffusion-webui-master\models\Lora\ghiles_2800.safetensors: AssertionError3<00:00, 11.78it/s]
Traceback (most recent call last):
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 279, in load_networks
    net = load_network(name, network_on_disk)
  File "C:\stable-diffusion-webui-master\extensions-builtin\Lora\networks.py", line 223, in load_network
    raise AssertionError(f"Could not find a module type (out of {', '.join([x.__class__.__name__ for x in module_types])}) that would accept those keys: {', '.join(weights.w)}")
AssertionError: Could not find a module type (out of ModuleTypeLora, ModuleTypeHada, ModuleTypeIa3, ModuleTypeLokr, ModuleTypeFull, ModuleTypeNorm, ModuleTypeGLora, ModuleTypeOFT) that would accept those keys: 0.attn1.to_k.lora_down.weight, 0.attn1.to_k.lora_up.weight, 0.attn1.to_out.0.lora_down.weight, 0.attn1.to_out.0.lora_up.weight, 0.attn1.to_q.lora_down.weight, 0.attn1.to_q.lora_up.weight, 0.attn1.to_v.lora_down.weight, 0.attn1.to_v.lora_up.weight, 0.attn2.to_k.lora_down.weight, 0.attn2.to_k.lora_up.weight, 0.attn2.to_out.0.lora_down.weight, 0.attn2.to_out.0.lora_up.weight, 0.attn2.to_q.lora_down.weight, 0.attn2.to_q.lora_up.weight, 0.attn2.to_v.lora_down.weight, 0.attn2.to_v.lora_up.weight

100%|██████████████████████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.14it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.45it/s]
Total progress: 100%|██████████████████████████████████████████████████████████████████| 40/40 [00:03<00:00, 11.56it/s]

Additional information

No response

klubkelli avatar Jan 30 '24 23:01 klubkelli

GitPython==3.1.32 Pillow==9.5.0 accelerate==0.21.0 basicsr==1.4.2 blendmodes==2022 clean-fid==0.1.35 einops==0.4.1 fastapi==0.94.0 gfpgan==1.3.8 gradio==3.41.2 httpcore==0.15 inflection==0.5.1 jsonmerge==1.8.0 kornia==0.6.7 lark==1.1.2 numpy==1.23.5 omegaconf==2.2.3 open-clip-torch==2.20.0 piexif==1.1.3 psutil==5.9.5 pytorch_lightning==1.9.4 realesrgan==0.3.0 resize-right==0.0.2 safetensors==0.3.1 scikit-image==0.21.0 timm==0.9.2 tomesd==0.1.3 torch torchdiffeq==0.2.3 torchsde==0.2.6 transformers==4.35.2 httpx==0.24.1

klubkelli avatar Jan 31 '24 02:01 klubkelli

Checking Dreambooth requirements... Installed version of accelerate: 0.21.0 [Dreambooth] accelerate v0.21.0 is already installed. Installed version of dadaptation: 3.2 [Dreambooth] dadaptation v3.2 is already installed. Installed version of diffusers: 0.25.0 [Dreambooth] diffusers v0.25.0 is already installed. Installed version of discord-webhook: 1.3.0 [Dreambooth] discord-webhook v1.3.0 is already installed. Installed version of fastapi: 0.94.0 [Dreambooth] fastapi is already installed. Installed version of gitpython: 3.1.32 [Dreambooth] gitpython v3.1.40 is not installed. Successfully installed gitpython-3.1.41

Installed version of pytorch_optimizer: 2.12.0 [Dreambooth] pytorch_optimizer v2.12.0 is already installed. Installed version of Pillow: 9.5.0 [Dreambooth] Pillow is already installed. Installed version of tqdm: 4.66.1 [Dreambooth] tqdm is already installed. Installed version of tomesd: 0.1.3 [Dreambooth] tomesd v0.1.2 is already installed. Installed version of tensorboard: 2.13.0 [Dreambooth] tensorboard v2.13.0 is already installed. [+] torch version 2.0.1+cu118 installed. [+] torchvision version 0.15.2+cu118 installed. [+] accelerate version 0.21.0 installed. [+] diffusers version 0.25.0 installed. [+] bitsandbytes version 0.41.2.post2 installed. [!] xformers NOT installed.

klubkelli avatar Jan 31 '24 16:01 klubkelli

Any suggestions?

klubkelli avatar Feb 02 '24 18:02 klubkelli

version: [v1.7.0] python: 3.10.6 torch: 2.0.1+cu118 xformers: N/A gradio: 3.41.2 checkpoint: [6ce0161689]

klubkelli avatar Feb 06 '24 05:02 klubkelli

This issue is stale because it has been open for 14 days with no activity. Remove stale label or comment or this will be closed in 30 days

github-actions[bot] avatar Feb 21 '24 00:02 github-actions[bot]

Same issue when using 731da11f9c800ac39e0a9a668d716ce76a151453 commit.

Mikhael-Danilov avatar Feb 26 '24 16:02 Mikhael-Danilov

Having similar issues, but I'll point you towards #857 #892 #897 which seem to cover this issue as well

NutritiousCookie avatar Mar 10 '24 00:03 NutritiousCookie

This issue is stale because it has been open for 14 days with no activity. Remove stale label or comment or this will be closed in 30 days

github-actions[bot] avatar Mar 24 '24 00:03 github-actions[bot]

this is issue https://github.com/d8ahazard/sd_dreambooth_extension/issues/1439 not the ones mentioned above

eija06 avatar Apr 06 '24 09:04 eija06

This issue is stale because it has been open for 14 days with no activity. Remove stale label or comment or this will be closed in 30 days

github-actions[bot] avatar Apr 22 '24 00:04 github-actions[bot]

This issue is still happening. Has anyone ever figured out how to fix this Dreambooth Lora bug?

klubkelli avatar May 06 '24 19:05 klubkelli

This issue is still happening. Has anyone ever figured out how to fix this Dreambooth Lora bug?

yes, but only by manually replacing the incorrect LORA keys mentioned in the error message to the correct ones, using a sed script on the output file. you can find the translation of the keys in this file: https://gist.github.com/eavae/a336029601f726737a0fd070f44c2f8a

But this is not a solution and I am getting no replies in #1439. My advice is to abandon this extension and use actively maintained software.

eija06 avatar May 09 '24 07:05 eija06

This issue is stale because it has been open for 14 days with no activity. Remove stale label or comment or this will be closed in 30 days

github-actions[bot] avatar May 24 '24 00:05 github-actions[bot]