fast-stable-diffusion icon indicating copy to clipboard operation
fast-stable-diffusion copied to clipboard

Error when trying to use BLIP, Dreambooth, Hypernetwork and Embeddings. SD 2.1 - Automatic1111 Updated.

Open artificialguybr opened this issue 2 years ago • 15 comments

First i have this problem when trying to use BLIP for Captions in Automatic1111

0% 0/770 [00:01<?, ?it/s] Error completing request Arguments: ('/content/gdrive/MyDrive/Dataset', '/content/gdrive/MyDrive/DatasetFinal', 768, 768, 'ignore', False, False, True, False, 0.5, 0.2, True, 0.9, 0.15, 0.5, False) {} Traceback (most recent call last): File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 45, in f res = list(func(*args, **kwargs)) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 28, in f res = func(*args, **kwargs) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/ui.py", line 19, in preprocess modules.textual_inversion.preprocess.preprocess(*args) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/preprocess.py", line 23, in preprocess preprocess_work(process_src, process_dst, process_width, process_height, preprocess_txt_action, process_flip, process_split, process_caption, process_caption_deepbooru, split_threshold, overlap_ratio, process_focal_crop, process_focal_crop_face_weight, process_focal_crop_entropy_weight, process_focal_crop_edges_weight, process_focal_crop_debug) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/preprocess.py", line 188, in preprocess_work save_pic(focal, index, params, existing_caption=existing_caption) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/preprocess.py", line 83, in save_pic save_pic_with_caption(image, index, params, existing_caption=existing_caption) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/preprocess.py", line 52, in save_pic_with_caption caption += shared.interrogator.generate_caption(image) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/interrogate.py", line 133, in generate_caption caption = self.blip_model.generate(gpu_image, sample=False, num_beams=shared.opts.interrogate_clip_num_beams, min_length=shared.opts.interrogate_clip_min_length, max_length=shared.opts.interrogate_clip_max_length) File "/content/gdrive/MyDrive/sd/stablediffusion/src/blip/models/blip.py", line 156, in generate outputs = self.text_decoder.generate(input_ids=input_ids, File "/usr/local/lib/python3.8/dist-packages/torch/autograd/grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 1268, in generate self._validate_model_kwargs(model_kwargs.copy()) File "/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 964, in _validate_model_kwargs raise ValueError( ValueError: The followingmodel_kwargs are not used by the model: ['encoder_hidden_states', 'encoder_attention_mask'] (note: typos in the generate arguments will also show up in this list)

I have this error when i'm trying to run Dreambooth in Automatic1111 Webui.

First part of the error when I try to start a new entry with Dreambooth installed on SD. (But it starts normally, only this error appears) `Error loading script: api.py Traceback (most recent call last): File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/scripts.py", line 184, in load_scripts module = script_loading.load_module(scriptfile.path) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/script_loading.py", line 13, in load_module exec(compiled, module.dict) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/api.py", line 10, in from extensions.sd_dreambooth_extension.dreambooth.sd_to_diff import extract_checkpoint File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/sd_to_diff.py", line 36, in from diffusers import ( ImportError: cannot import name 'HeunDiscreteScheduler' from 'diffusers' (/usr/local/lib/python3.8/dist-packages/diffusers/init.py)

Error loading script: main.py Traceback (most recent call last): File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/scripts.py", line 184, in load_scripts module = script_loading.load_module(scriptfile.path) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/script_loading.py", line 13, in load_module exec(compiled, module.dict) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/main.py", line 5, in from extensions.sd_dreambooth_extension.dreambooth.diff_to_sd import compile_checkpoint File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/diff_to_sd.py", line 214, in def convert_text_enc_state_dict_v20(text_enc_dict: dict[str, torch.Tensor]): TypeError: 'type' object is not subscriptable

LatentDiffusion: Running in v-prediction mode DiffusionWrapper has 865.91 M params. Loading weights [6bccbcc6] from /content/gdrive/MyDrive/sd/stable-diffusion-webui/models/Stable-diffusion/model.ckpt Applying xformers cross attention optimization. Model loaded. Loaded a total of 0 textual inversion embeddings. Embeddings: `

Then when I go into the GUI, it doesn't show the Dreambooth tab.

artificialguybr avatar Dec 07 '22 21:12 artificialguybr

I tried embedding train and got this error:

File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding loss = shared.sd_model(x, c)[0] / gradient_step File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/models/diffusion/ddpm.py", line 846, in forward return self.p_losses(x, c, t, *args, **kwargs) File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/models/diffusion/ddpm.py", line 903, in p_losses logvar_t = self.logvar[t].to(self.device) RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

artificialguybr avatar Dec 08 '22 05:12 artificialguybr

And this error with hypernetworks

Traceback (most recent call last): File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 45, in f res = list(func(*args, **kwargs)) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/call_queue.py", line 28, in f res = func(*args, **kwargs) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/hypernetworks/ui.py", line 50, in train_hypernetwork hypernetwork, filename = modules.hypernetworks.hypernetwork.train_hypernetwork(*args) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/hypernetworks/hypernetwork.py", line 444, in train_hypernetwork hypernetwork.train_mode() AttributeError: 'Hypernetwork' object has no attribute 'train_mode'

artificialguybr avatar Dec 08 '22 05:12 artificialguybr

I'm having the same issue with embeddings. Worked fine 2 days ago. I deleted the sd folder and reinstalled, but still getting the same error:

File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/textual_inversion/textual_inversion.py", line 328, in train_embedding loss = shared.sd_model(x, c)[0] / gradient_step File "/usr/local/lib/python3.8/dist-packages/torch/nn/modules/module.py", line 1190, in _call_impl return forward_call(*input, **kwargs) File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/models/diffusion/ddpm.py", line 846, in forward return self.p_losses(x, c, t, *args, **kwargs) File "/content/gdrive/MyDrive/sd/stablediffusion/ldm/models/diffusion/ddpm.py", line 903, in p_losses logvar_t = self.logvar[t].to(self.device) RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cpu)

7wpanc24 avatar Dec 08 '22 12:12 7wpanc24

the errors come from files untouched by the colab, so you might need to open an issue in A1111 repo

TheLastBen avatar Dec 08 '22 12:12 TheLastBen

can do. thank you

7wpanc24 avatar Dec 08 '22 12:12 7wpanc24

@TheLastBen I was able to fix Hypernetworks and Embedding using this fix.

https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/5523#issuecomment-1343041303 I've added self.logvar = self.logvar.to(self.device) in ddpm.py above line 903 before starting anything.

However, when it goes to generate the example images, they turn black. The normal generation in txt2img is normal. In the automatic's repo it says that:

If 2.0 or 2.1 is generating black images, enable full precision with --no-half or try using the --xformers optimization.

However, I do not know how to add this to the part only to the generation of previews of Hypernetworks and Embedding. You know and could help me?

artificialguybr avatar Dec 10 '22 05:12 artificialguybr

I also got the same cannot import name 'HeunDiscreteScheduler' from 'diffusers' error message after restarting the WebUI after installing the Dreambooth extension. This results in no Dreambooth tab being shown.

~I get rid of the error if I run export REQS_FILE="./extensions/sd_dreambooth_extension/requirements.txt" before the last cell in the Colab notebook (i.e. Start stable-diffusion) BUT then I get another error:~

~Error loading script: api.py Traceback (most recent call last): File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/scripts.py", line 195, in load_scripts module = script_loading.load_module(scriptfile.path) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/script_loading.py", line 13, in load_module exec(compiled, module.dict) File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/scripts/api.py", line 9, in from extensions.sd_dreambooth_extension.dreambooth import dreambooth File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/dreambooth.py", line 12, in from diffusers.utils import logging as dl File "/usr/local/lib/python3.8/dist-packages/diffusers/init.py", line 32, in from .pipelines import ( ModuleNotFoundError: No module named 'diffusers.pipelines'~

EDIT: I still get the same error as OP.

roperi avatar Dec 13 '22 14:12 roperi

@jvkap

EDIT: The following solved the 'Dreambooth tab not showing in the web UI' problem.

I solved the HeunDiscreteScheduler error by !pip install git+https://github.com/huggingface/diffusers. This should install a compatible version of Diffusers and fix the error.

But if then throws a transformers error do !pip install git+https://github.com/huggingface/transformers. This should install a compatible version of Transformers.

Having done that you should be left with one last error message :

def convert_text_enc_state_dict_v20(text_enc_dict: dict[str, torch.Tensor]):
TypeError: 'type' object is not subscriptable

This seem to be a Python3.8 problem (i think doesn't happen with Python 3.11). So open the file "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/diff_to_sd.py" and then delete the type hint from the parameter of the function so change def convert_text_enc_state_dict_v20(text_enc_dict: dict[str, torch.Tensor]): to def convert_text_enc_state_dict_v20(text_enc_dict). Also change def convert_text_enc_state_dict(text_enc_dict: dict[str, torch.Tensor]) to def convert_text_enc_state_dict(text_enc_dict):. These two deletions are 100% safe.

Save those two edited files, re-run everything and voila! You should finally get the Dreambooth tab.

PS. I'm using model 1.5 but this should work for 2.X

roperi avatar Dec 13 '22 17:12 roperi

pip install git+https://github.com/huggingface/diffusers

GitHub has apparently discontinued HTTPS authentication, so watch out for that. I had to use…

pip install git+ssh://[email protected]/huggingface/diffusers

thinkyhead avatar Dec 14 '22 04:12 thinkyhead

Thanks for the heads up, @thinkyhead.

I encountered another problem when training in Dreambooth. And again the error is thrown because of the type hint in the main function in the train_dreambooth.py file. So go to the Dreambooth extensions folder in your stable-diffusion directory, and look for the train_dreambooth.py file. Open it and change def main(args: DreamboothConfig, memory_record, use_subdir, lora_model=None, lora_alpha=1) -> tuple[DreamboothConfig, dict, str]: to def main(args: DreamboothConfig, memory_record, use_subdir, lora_model=None, lora_alpha=1):.

roperi avatar Dec 14 '22 11:12 roperi

Ok, I won't pretend this is perfect. Seems to allow training to start for me, just testing it now. But this might allow someone to do a quick copy paste into existing Colab.

Based on the info in here, I've created two additional sections.

#@markdown # Tweaks #1
#@markdown Some tweaks to enable some Automatic1111 extensions, installing updated diffusers and transformers.

#@markdown Also importing *subprocess with getoutput*
from subprocess import getoutput

!pip install git+https://github.com/huggingface/diffusers

!pip install git+https://github.com/huggingface/transformers
#@markdown # Tweaks #2
#@markdown Some tweaks to enable some Automatic1111 extensions
#@markdown from the discussion:

#@markdown https://github.com/TheLastBen/fast-stable-diffusion/issues/914
remove_xformers = False #@param {type:"boolean"}
#@markdown  - Remove xFormers? Only remove if required.

if remove_xformers:
  !pip uninstall xformers

replace_diff_to_sd = False #@param {type:"boolean"}
#@markdown  - Replace diff_to_sd.py within dreambooth extension

replace_train_dreambooth = False #@param {type:"boolean"}
#@markdown  - Replace train_dreambooth.py within dreambooth extension

if replace_diff_to_sd:
  !wget https://raw.githubusercontent.com/mediocreatmybest/gaslightingeveryone/main/Scripts/scraps/StableDiffusion-misc/diff_to_sd.py -O /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/diff_to_sd.py

if replace_train_dreambooth:
  !wget https://raw.githubusercontent.com/mediocreatmybest/gaslightingeveryone/main/Scripts/scraps/StableDiffusion-misc/train_dreambooth.py -O /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/train_dreambooth.py


reset_dreambooth_ext = False #@param {type:"boolean"}
#@markdown  - Removes modified files and updates dreambooth extension

if reset_dreambooth_ext:
  with capture.capture_output() as cap:
    !rm /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/diff_to_sd.py  
    !rm /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/train_dreambooth.py
    %cd /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension
    print('[1;32m')
    !git pull
    clear_output()
    print('[1;32mDONE !')

I did upload the modification if it doesn't display here correctly.

https://github.com/mediocreatmybest/gaslightingeveryone/blob/main/Scripts/scraps/StableDiffusion-misc/tweaked_fast_stable_diffusion_AUTOMATIC1111.ipynb

mediocreatmybest avatar Dec 15 '22 02:12 mediocreatmybest

Did a training session for 10,000 steps with LORA.

Got the following at the end.


Traceback (most recent call last):
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/train_dreambooth.py", line 977, in main
    for step, batch in enumerate(train_dataloader):
  File "/usr/local/lib/python3.8/dist-packages/accelerate/data_loader.py", line 357, in __iter__
    next_batch = next(dataloader_iter)
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 628, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/dataloader.py", line 671, in _next_data
    data = self._dataset_fetcher.fetch(index)  # may raise StopIteration
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in fetch
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/usr/local/lib/python3.8/dist-packages/torch/utils/data/_utils/fetch.py", line 58, in <listcomp>
    data = [self.dataset[idx] for idx in possibly_batched_index]
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/dreambooth/SuperDataset.py", line 257, in __getitem__
    concept_data = self.concepts[c_index]
IndexError: list index out of range
|=======================================

I also can't see if or where any models were saved?

mediocreatmybest avatar Dec 15 '22 04:12 mediocreatmybest

saved?

Ok, did a quick check compared to my local install. It did train for the full 10,000 steps but didn't actually save any checkpoints or pt files etc

mediocreatmybest avatar Dec 15 '22 11:12 mediocreatmybest

@mediocreatmybest

Thanks! Very thankful for your notebook update.

I also had Dreambooth training for an hour just for the last moment to throw an error. Didn't save anything.

In my case I needed an updated version of accelerate. So I'd be great if you can add this one line (as opposed to the two lines you added based on my previous suggestion):

!pip install -U diffusers transformers accelerate

roperi avatar Dec 15 '22 21:12 roperi

@mediocreatmybest

Thanks! Very thankful for your notebook update.

I also had Dreambooth training for an hour just for the last moment to throw an error. Didn't save anything.

In my case I needed an updated version of accelerate. So I'd be great if you can add this one line (as opposed to the two lines you added based on my previous suggestion):

!pip install -U diffusers transformers accelerate

No worries, I've updated the file but I'm getting a different error now.

Returning result: Exception training model: main() got an unexpected keyword argument 'lora_txt_alpha'

Not sure if that is related to the modified files or not for dreambooth?

mediocreatmybest avatar Dec 16 '22 00:12 mediocreatmybest

line

Ok, this is pretty wonky and xformers is super unreliable to build and distribute via a wheel. I've compiled it about 4 times to build a wheel but it never detects it when the colab is restarted.

Only way I could get it to work was via GitHub build and install with pip, takes over an hour to run, so I don't think it's worth it.

Anyway, totally unsupported and no doubt could break.

Might be useful to someone.

Behold the modified colab based on fast stable diffusion with Python 3.10 and Automatic1111, Cludge Edition

https://github.com/mediocreatmybest/gaslightingeveryone/blob/main/Colab/fast_stable_diffusion_AUTOMATIC1111_CludgeEdition.ipynb

Results may vary.

mediocreatmybest avatar Dec 17 '22 09:12 mediocreatmybest

line

Ok, this is pretty wonky and xformers is super unreliable to build and distribute via a wheel. I've compiled it about 4 times to build a wheel but it never detects it when the colab is restarted.

Only way I could get it to work was via GitHub build and install with pip, takes over an hour to run, so I don't think it's worth it.

Anyway, totally unsupported and no doubt could break.

Might be useful to someone.

Behold the modified colab based on fast stable diffusion with Python 3.10 and Automatic1111, Cludge Edition

https://github.com/mediocreatmybest/gaslightingeveryone/blob/main/Colab/fast_stable_diffusion_AUTOMATIC1111_CludgeEdition.ipynb

Results may vary.

Xformers fixed. --pre now available. Only just saw it.

mediocreatmybest avatar Dec 17 '22 11:12 mediocreatmybest

@mediocreatmybest:

I haven't messed with LORA yet so I don't know why you are getting the error message. I didn't get any and I have Dreambooth trained two models with success. Did you fine tune anything with LORA?

Your error message: Returning result: Exception training model: main() got an unexpected keyword argument 'lora_txt_alpha

...seems to be related to an old version of whatever file has that main() function. Perhaps you could try pip installing (with -U) its related package and load this function in the Colab Notebook and run it with the same keyword argument and see if it still throws the error before doing a hours-long training.

EDIT: Sorry, didn't notice you already told us you fine tuned using LORA.

roperi avatar Dec 17 '22 12:12 roperi

@mediocreatmybest:

I haven't messed with LORA yet so I don't know why you are getting the error message. I didn't get any and I have Dreambooth trained two models with success. Did you fine tune anything with LORA?

Your error message: Returning result: Exception training model: main() got an unexpected keyword argument 'lora_txt_alpha

...seems to be related to an old version of whatever file has that main() function. Perhaps you could try pip installing (with -U) its related package and load this function in the Colab Notebook and run it with the same keyword argument and see if it still throws the error before doing a hours-long training.

EDIT: Sorry, didn't notice you already told us you fine tuned using LORA.

Yeah seems to be related to LORA and SDv2. I've got it working correctly with SDv1 andSDv2 no worries without LORA. As mentioned above, I got python3.10/Xformers working on that Cludged colab I posted above. No need to mod the files to get it working with 3.8. Not perfect, and might be a better way to get it to work. Works for me now quite well though.

mediocreatmybest avatar Dec 17 '22 13:12 mediocreatmybest

@mediocreatmybest

Amazing! I'll test it and get back if anything goes wrong. Thanks!

roperi avatar Dec 17 '22 13:12 roperi

@mediocreatmybest: I haven't messed with LORA yet so I don't know why you are getting the error message. I didn't get any and I have Dreambooth trained two models with success. Did you fine tune anything with LORA? Your error message: Returning result: Exception training model: main() got an unexpected keyword argument 'lora_txt_alpha ...seems to be related to an old version of whatever file has that main() function. Perhaps you could try pip installing (with -U) its related package and load this function in the Colab Notebook and run it with the same keyword argument and see if it still throws the error before doing a hours-long training. EDIT: Sorry, didn't notice you already told us you fine tuned using LORA.

Yeah seems to be related to LORA and SDv2. I've got it working correctly with SDv1 andSDv2 no worries without LORA. As mentioned above, I got python3.10/Xformers working on that Cludged colab I posted above. No need to mod the files to get it working with 3.8. Not perfect, and might be a better way to get it to work. Works for me now quite well though.

Lora was already made for SD 2.1. You can use an exclusive Colab for LORA here.

https://colab.research.google.com/drive/1iSFDpRBKEWr2HLlz243rbym3J2X95kcy?usp=sharing#scrollTo=RXhqKsN8cEop

artificialguybr avatar Dec 18 '22 06:12 artificialguybr

@mediocreatmybest:

I haven't messed with LORA yet so I don't know why you are getting the error message. I didn't get any and I have Dreambooth trained two models with success. Did you fine tune anything with LORA?

Your error message: Returning result: Exception training model: main() got an unexpected keyword argument 'lora_txt_alpha

...seems to be related to an old version of whatever file has that main() function. Perhaps you could try pip installing (with -U) its related package and load this function in the Colab Notebook and run it with the same keyword argument and see if it still throws the error before doing a hours-long training.

EDIT: Sorry, didn't notice you already told us you fine tuned using LORA.

Yeah seems to be related to LORA and SDv2. I've got it working correctly with SDv1 andSDv2 no worries without LORA. As mentioned above, I got python3.10/Xformers working on that Cludged colab I posted above. No need to mod the files to get it working with 3.8. Not perfect, and might be a better way to get it to work. Works for me now quite well though.

Lora was already made for SD 2.1.

You can use an exclusive Colab for LORA here.

https://colab.research.google.com/drive/1iSFDpRBKEWr2HLlz243rbym3J2X95kcy?usp=sharing#scrollTo=RXhqKsN8cEop

Thanks! Yeah this was an attempt to use the AUTOMATIC1111 colab with Lora since it's built into the dreambooth extension, so I suspect it could be an issue with how it's implemented in the extension, since I get the same error on my local install.

mediocreatmybest avatar Dec 18 '22 07:12 mediocreatmybest

line

Ok, this is pretty wonky and xformers is super unreliable to build and distribute via a wheel. I've compiled it about 4 times to build a wheel but it never detects it when the colab is restarted.

Only way I could get it to work was via GitHub build and install with pip, takes over an hour to run, so I don't think it's worth it.

Anyway, totally unsupported and no doubt could break.

Might be useful to someone.

Behold the modified colab based on fast stable diffusion with Python 3.10 and Automatic1111, Cludge Edition

https://github.com/mediocreatmybest/gaslightingeveryone/blob/main/Colab/fast_stable_diffusion_AUTOMATIC1111_CludgeEdition.ipynb

Results may vary.

You may just install xformer fromhere : !pip install https://github.com/camenduru/stable-diffusion-webui-colab/releases/download/0.0.15/xformers-0.0.15+f82722f.d20221217-cp310-cp310-linux_x86_64.whl Every person doesnt have to build. one may built it on same type of OS and python version and make .whl file.

UsamaKenway avatar Dec 19 '22 10:12 UsamaKenway

line

Ok, this is pretty wonky and xformers is super unreliable to build and distribute via a wheel. I've compiled it about 4 times to build a wheel but it never detects it when the colab is restarted.

Only way I could get it to work was via GitHub build and install with pip, takes over an hour to run, so I don't think it's worth it.

Anyway, totally unsupported and no doubt could break.

Might be useful to someone.

Behold the modified colab based on fast stable diffusion with Python 3.10 and Automatic1111, Cludge Edition

https://github.com/mediocreatmybest/gaslightingeveryone/blob/main/Colab/fast_stable_diffusion_AUTOMATIC1111_CludgeEdition.ipynb

Results may vary.

You may just install xformer fromhere :

!pip install https://github.com/camenduru/stable-diffusion-webui-colab/releases/download/0.0.15/xformers-0.0.15+f82722f.d20221217-cp310-cp310-linux_x86_64.whl

Every person doesnt have to build. one may built it on same type of OS and python version and make .whl file.

Thanks! Xformers also have a pre version out now on pip. pip install --pre xformers

In the script I updated it to that version, seems to run fine. 👍

mediocreatmybest avatar Dec 19 '22 11:12 mediocreatmybest

Ok. So we fixed the Textual Inversion part, but we're still buggy in Lora and Dreambooth. Is that correct?

It would be interesting to have help from @d8ahazard for that Dreambooth issue.

And @cloneofsimo for Lora problem.

Would anyone be interested in opening an issue in their repo?

artificialguybr avatar Dec 19 '22 13:12 artificialguybr

Wait what? I didn't know this even existed. You guys have adapted LoRA here?

cloneofsimo avatar Dec 19 '22 13:12 cloneofsimo

Wait what? I didn't know this even existed. You guys have adapted LoRA here?

Yep! d8ahazard has it implemented in his dreambooth extension. I've successfully gotten it working with SD1.x but it keeps crashing with SD2.x, so I'm not sure if it is something either A) I'm doing incorrectly, B) My dodgy Colab, or C) something not working correctly in the extension. I honestly haven't flagged it as an issue until I can test it on a local install. My local laptop doesn't obviously have enough vRAM even with an NVIDIA GPU.

mediocreatmybest avatar Dec 19 '22 13:12 mediocreatmybest

Ok. So we fixed the Textual Inversion part, but we're still buggy in Lora and Dreambooth. Is that correct?

It would be interesting to have help from @d8ahazard for that Dreambooth issue.

And @cloneofsimo for Lora problem.

Would anyone be interested in opening an issue in their repo?

Dreambooth works for me but only after making the changes I suggested above.

roperi avatar Dec 19 '22 16:12 roperi

@mediocreatmybest

Trying out your script and I am getting this error:

Error running install.py for extension /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension.
Command: "/usr/bin/python3" "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/install.py"
Error code: 1
stdout: <empty>
stderr: Traceback (most recent call last):
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd_dreambooth_extension/install.py", line 10, in <module>
    from launch import run
ModuleNotFoundError: No module named 'launch'

Seems to be failing here: https://github.com/d8ahazard/sd_dreambooth_extension/blob/main/install.py#L10

I have no idea what that package is suppose to be and it is not part of the python stdlib. Do you see that line of code in your google drive? Are you using an older version of the extension?

ekeric13 avatar Dec 20 '22 01:12 ekeric13

Hey all!

So, I think the issue here (at least from the first post) is that the latest transformers version has some check that throws an issue if it finds extra kwargs.

"/usr/local/lib/python3.8/dist-packages/transformers/generation_utils.py", line 964, in _validate_model_kwargs raise ValueError( ValueError: The followingmodel_kwargs are not used by the model: ['encoder_hidden_states', 'encoder_attention_mask'] (note: typos in the generate arguments will also show up in this list)

I had that patched out using the "xattention.py" file in my repo, which is just sort of a generic catchall for monkey-patching various bugs I found along the way. But, I bumped my diffusers and transformers versions to match the official diffusers repo (as of ver 0.10.2), and didn't have to monkey-patch this any more. So, I think the right answer is "check the diffusers and transfomers" versions.

If I missed some other issue in the discussion, throw me an error message and I'll let you know if it's anything else I hacked in to fix stuff.

@ekeric13 - "Launch" refers to launch.py from the auto1111 script. It's just a wrapper for the run_command call, which I should probably just remove and add my own, as I'm already working to make less of the extension depend directly on Auto1111, just for compatibility and sanity's sake. ;)

d8ahazard avatar Dec 20 '22 02:12 d8ahazard