[Bug]: LORA's while using INPAINTING mode completely broken after latest commits

[Akis-M]

Is there an existing issue for this?

• [X] I have searched the existing issues and checked the recent builds/commits

What happened?

I can still generate images, but it seems that Loras (when in INPAINTING mode) are completely ignored no matter what. And I see this error in the log every time I try to generate:

activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x0000019D8A7E37F0>, <modules.extra_networks.ExtraNetworkParams object at 0x0000019D8A648C40>]: RuntimeError Traceback (most recent call last): File "C:\StableDif\stable-diffusion-webui\modules\extra_networks.py", line 75, in activate extra_network.activate(p, extra_network_args) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate lora.load_loras(names, multipliers) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 213, in load_loras lora = load_lora(name, lora_on_disk.filename) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 175, in load_lora module.weight.copy_(weight) RuntimeError: output with shape [4, 320, 1, 1] doesn't match the broadcast shape [4, 320, 3, 3]

Steps to reproduce the problem

1. Go to inpainting, inpaint what you want, enter your prompt, select any lora
2. Generate
3. Get picture with no Lora applied, and error in log

What should have happened?

1. Go to inpainting, inpaint what you want, enter your prompt, select any lora
2. Generate
3. Get picture influenced by lora

Commit where the problem happens

I first noticed the issue after 4c1ad743e3baf1246db0711aa0107debf036a12b but after looking at the commits and spamming git reset --hard one by one, the first commit where the issue appears (and persists in every single commit up to today) is:

650ddc9dd3c1d126221682be8270f7fba1b5b6ce

What platforms do you use to access the UI ?

Windows

What browsers do you use to access the UI ?

Google Chrome

Command Line Arguments

No

List of extensions

Stable-Diffusion-Webui-Civitai-Helper |   |   |  -- | -- | -- | -- depthmap2mask | https://github.com/Extraltodeus/depthmap2mask.git | 7eedf6af (Mon Mar 20 10:16:05 2023) | latest sd-webui-controlnet | https://github.com/Mikubill/sd-webui-controlnet.git | 241c05f8 (Thu Mar 23 15:18:35 2023) | latest sd-webui-supermerger | https://github.com/hako-mikan/sd-webui-supermerger | 4c6a3650 (Mon Mar 20 11:41:02 2023) | latest stable-diffusion-webui-conditioning-highres-fix | https://github.com/klimaleksus/stable-diffusion-webui-conditioning-highres-fix.git | e01bfcf4 (Wed Jan 4 18:39:42 2023) | latest

Console logs

To create a public link, set share=True in launch(). Startup time: 20.1s (import torch: 3.2s, import gradio: 1.9s, import ldm: 0.7s, other imports: 2.0s, list SD models: 0.4s, setup codeformer: 0.1s, load scripts: 4.1s, load SD checkpoint: 6.6s, create ui: 1.0s, gradio launch: 0.1s). 100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00, 4.84it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 5.93it/s] activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x0000018F57E5AA10>, <modules.extra_networks.ExtraNetworkParams object at 0x0000018F57E5AC20>]: RuntimeError Traceback (most recent call last): File "C:\StableDif\stable-diffusion-webui\modules\extra_networks.py", line 75, in activate extra_network.activate(p, extra_network_args) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate lora.load_loras(names, multipliers) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 214, in load_loras lora = load_lora(name, lora_on_disk.filename) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 176, in load_lora module.weight.copy_(weight) RuntimeError: output with shape [4, 320, 1, 1] doesn't match the broadcast shape [4, 320, 3, 3]

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 5.51it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.08it/s] activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x0000018F57E23280>, <modules.extra_networks.ExtraNetworkParams object at 0x0000018F57E22F50>]: RuntimeError Traceback (most recent call last): File "C:\StableDif\stable-diffusion-webui\modules\extra_networks.py", line 75, in activate extra_network.activate(p, extra_network_args) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate lora.load_loras(names, multipliers) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 214, in load_loras lora = load_lora(name, lora_on_disk.filename) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 176, in load_lora module.weight.copy_(weight) RuntimeError: output with shape [4, 320, 1, 1] doesn't match the broadcast shape [4, 320, 3, 3]

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.12it/s] activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x0000018F58361570>, <modules.extra_networks.ExtraNetworkParams object at 0x0000018F58360550>]: RuntimeError Traceback (most recent call last): File "C:\StableDif\stable-diffusion-webui\modules\extra_networks.py", line 75, in activate extra_network.activate(p, extra_network_args) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate lora.load_loras(names, multipliers) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 214, in load_loras lora = load_lora(name, lora_on_disk.filename) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 176, in load_lora module.weight.copy_(weight) RuntimeError: output with shape [4, 320, 1, 1] doesn't match the broadcast shape [4, 320, 3, 3]

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.13it/s] activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x0000018F583701C0>, <modules.extra_networks.ExtraNetworkParams object at 0x0000018F4EA0D390>]: RuntimeError Traceback (most recent call last): File "C:\StableDif\stable-diffusion-webui\modules\extra_networks.py", line 75, in activate extra_network.activate(p, extra_network_args) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate lora.load_loras(names, multipliers) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 214, in load_loras lora = load_lora(name, lora_on_disk.filename) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 176, in load_lora module.weight.copy_(weight) RuntimeError: output with shape [4, 320, 1, 1] doesn't match the broadcast shape [4, 320, 3, 3]

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.07it/s] activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x0000018F583700D0>, <modules.extra_networks.ExtraNetworkParams object at 0x0000018F58362E60>]: RuntimeError Traceback (most recent call last): File "C:\StableDif\stable-diffusion-webui\modules\extra_networks.py", line 75, in activate extra_network.activate(p, extra_network_args) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate lora.load_loras(names, multipliers) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 214, in load_loras lora = load_lora(name, lora_on_disk.filename) File "C:\StableDif\stable-diffusion-webui\extensions-builtin\Lora\lora.py", line 176, in load_lora module.weight.copy_(weight) RuntimeError: output with shape [4, 320, 1, 1] doesn't match the broadcast shape [4, 320, 3, 3]

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 5.99it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 80/80 [00:14<00:00, 5.70it/s] Total progress: 100%|██████████████████████████████████████████████████████████████████| 80/80 [00:14<00:00, 6.01it/s]

Additional information

No response

[mart-hill]

I noticed the embeddings are also being totally ignored (no error though), and generation of the image never happens, once an embedding is called in the prompt (or neg one). It seems the generation of the image is being quietly queued, but never executed nor any progress is being shown. Next generations won't be possible unless total WebUI restart occurs.

[merecesarchviz]

same here after update today the SD!

[fractal-fumbler]

@Giganoob23, what torch version do you have installed: 1.13.1 or 2.0.0?

maybe relevant https://github.com/AUTOMATIC1111/stable-diffusion-webui/discussions/8485

test if LoRA is working https://civitai.com/models/11843/oil-impressionist-painterly-style is

[bismark211]

the selection of the None section is missing in the lora and in the hypernetwork, by selecting the lora, you can remove it only by editing the ui-configand rebooting

[Blackened15]

Same here, prompt produced the same image with and without the <lora:randomlora_v1:1> tag.

Seems I mistakenly tested using a 2.1 model.

[Akis-M]

@Giganoob23, what torch version do you have installed: 1.13.1 or 2.0.0?

maybe relevant #8485

test if LoRA is working https://civitai.com/models/11843/oil-impressionist-painterly-style is

@fractal-fumbler I updated the issue after I observed that this error only appears in INPAINTING mode. Normal txt2img generations do apply Lora's, but inpainting fails to call the lora, with the errors I attached. I also updated to indicate that this still happens even in the latest commits.

Also including here as an issue, the fact that in the latest UI you can no longer DESELECT a lora after picking one and the only way to clear the selection is restarting the UI which is quite troublesome.

My torch is 1.12.1+cu116 same one i've had for a while now.

[Juafeis]

I have problems with both txt2img and img2img inpainting

[roiniti]

Same problem here, i have the same model and the same lora that i used days ago, not working anymore, RuntimeError: output with shape

[TricksyZerg]

I also have this problem with loras, happens in txt2img and img2img inpainting

EDIT: it seems like the bug only happens when I click the Lora through the "Extra Networks" tab, not when I type it out

[nasiri]

I have quick fixed at least the problem for myself by adding following code in the style.css. Quick reminder, I had added sd_lora to my quicklist UI, so the code above is for deselecting it there. You would need to adjust the Class or ID accordingly for the Extra Networks Tab i guess. It also allows to fix the "Weight Error" or "RuntimeError: output with shape" with deselecting any SD1.5 Loras which accidently got enabled by default for an SD2.1 Checkpopint.

.gradio-dropdown .token-remove.remove-all.remove-all{
    display: none;
}

add the following code in the style.css around Line 100 the class for the sd_lora seems to be .svelte-a6vu2r in my webui, i am not sure if thats a unique identifier or is generally that same for all of us. If the REMOVE (X) button doesnt reappear, check the Classname for the container in the html code of the UI.

.svelte-a6vu2r .token-remove{
    display: block !important;
}

... so at least the X Button is visible again, so you can uncheck the Lora which is always automaticly selected. The initial problem is that a "none" selecting is missing completly with the sd_lora dropdown. We would need a fix in that regard.

[Fuciuss]

Stuck here too

[Akis-M]

I have quick fixed at least the problem for myself by adding following code in the style.css. Quick reminder, I had added sd_lora to my quicklist UI, so the code above is for deselecting it there. You would need to adjust the Class or ID accordingly for the Extra Networks Tab i guess. It also allows to fix the "Weight Error" or "RuntimeError: output with shape" with deselecting any SD1.5 Loras which accidently got enabled by default for an SD2.1 Checkpopint.

.gradio-dropdown .token-remove.remove-all.remove-all{
    display: none;
}

add the following code in the style.css around Line 100 the class for the sd_lora seems to be .svelte-a6vu2r in my webui, i am not sure if thats a unique identifier or is generally that same for all of us. If the REMOVE (X) button doesnt reappear, check the Classname for the container in the html code of the UI.

.svelte-a6vu2r .token-remove{
    display: block !important;
}

... so at least the X Button is visible again, so you can uncheck the Lora which is always automaticly selected. The initial problem is that a "none" selecting is missing completly with the sd_lora dropdown. We would need a fix in that regard.

@nasiri Tried both of your codes. I can deselect any lora/extra network but they don't fix the RuntimeError: output with shape [4, 320, 1, 1] doesn't match the broadcast shape [4, 320, 3, 3] error. Which is the real issue as LORAs are completely unuseable in inpainting right now.

I'm using 1.5 LORA's with 1.5 model. Loras that worked completely fine before this commit. Wtf?

After manually looking around at the commit history around when I first noticed this issue and spamming a bunch of git reset --hard, I found the exact commit where the issue appears. As someone who's not a programmer this is all I can do and I hope someone can revert or fix that commit.

650ddc9dd3c1d126221682be8270f7fba1b5b6ce

Can someone have a look?

[noimjosh]

My self-trained SD 1.5 LORAs were failing in TXT2IMG with the output shape error after updating to the latest commits on 3/30 until I reverted to before the 650ddc9dd3c1d126221682be8270f7fba1b5b6ce commit as @Giganoob23 mentioned

[joaoescribano]

I have quick fixed at least the problem for myself by adding following code in the style.css. Quick reminder, I had added sd_lora to my quicklist UI, so the code above is for deselecting it there. You would need to adjust the Class or ID accordingly for the Extra Networks Tab i guess. It also allows to fix the "Weight Error" or "RuntimeError: output with shape" with deselecting any SD1.5 Loras which accidently got enabled by default for an SD2.1 Checkpopint.

.gradio-dropdown .token-remove.remove-all.remove-all{
    display: none;
}

add the following code in the style.css around Line 100 the class for the sd_lora seems to be .svelte-a6vu2r in my webui, i am not sure if thats a unique identifier or is generally that same for all of us. If the REMOVE (X) button doesnt reappear, check the Classname for the container in the html code of the UI.

.svelte-a6vu2r .token-remove{
    display: block !important;
}

... so at least the X Button is visible again, so you can uncheck the Lora which is always automaticly selected. The initial problem is that a "none" selecting is missing completly with the sd_lora dropdown. We would need a fix in that regard.

As the repo it isn't fixed yet, your fix made possible to remove the option from the settings. (I live edited the css style in chrome developer tools, AKA F12) Thank you

[suede299]

activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x000002621605A2C0>, <modules.extra_networks.ExtraNetworkParams object at 0x000002621605AB90>]: RuntimeError Traceback (most recent call last): File "D:\suede\Desktop\SDWegUI\modules\extra_networks.py", line 75, in activate extra_network.activate(p, extra_network_args) File "D:\suede\Desktop\SDWegUI\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate lora.load_loras(names, multipliers) File "D:\suede\Desktop\SDWegUI\extensions-builtin\Lora\lora.py", line 214, in load_loras lora = load_lora(name, lora_on_disk.filename) File "D:\suede\Desktop\SDWegUI\extensions-builtin\Lora\lora.py", line 176, in load_lora module.weight.copy_(weight) RuntimeError: output with shape **[64, 320]** doesn't match the broadcast shape **[64, 320, 64, 320]**

Mine looks even more outrageous, with 2 bits missing straight away. 2.1 sd model But using the lora-add extension, it works. Hopefully it will be fixed soon, because it needs to be used with another lora weight extension, which can only be used by filling in the prompt. -Translated by deepl

[fritty]

Just updated webui yesterday, and today found out that some of my Loras dont work. Copied this fix: https://github.com/AUTOMATIC1111/stable-diffusion-webui/pull/9504/commits/538600288cb3169c7771dd18b703f89d24fb1d88 It worked

[Akis-M]

@fritty

Yeah but that pull request has not been approved yet so even if you update the webui it's not gonna be in it. How do you "copy" the fix in

[fritty]

@Giganoob23 Literally just copy those changes to lora.py

[robproctor]

I just patched my file and it worked, everything seems to work now. However, I would like to understand what the patch is doing exactly if someone can elaborate? Will this affect things in some negative way? Slower? Less accurate? I could run tests, but figured I would ask first.

Thanks

[cooperdk]

Not working for me, I trained a lycon myself which still doesnt work.

[robproctor]

@cooperdk Which version are you on? And, are you getting errors in console? Do any work? Before I patched some would work, but some would not. After patch they all worked. I am also not on latest. My commit is: 22bcc7be

[hyyuan123]

activating extra network lora with arguments [<modules.extra_networks.ExtraNetworkParams object at 0x000002621605A2C0>, <modules.extra_networks.ExtraNetworkParams object at 0x000002621605AB90>]: RuntimeError Traceback (most recent call last): File "D:\suede\Desktop\SDWegUI\modules\extra_networks.py", line 75, in activate extra_network.activate(p, extra_network_args) File "D:\suede\Desktop\SDWegUI\extensions-builtin\Lora\extra_networks_lora.py", line 23, in activate lora.load_loras(names, multipliers) File "D:\suede\Desktop\SDWegUI\extensions-builtin\Lora\lora.py", line 214, in load_loras lora = load_lora(name, lora_on_disk.filename) File "D:\suede\Desktop\SDWegUI\extensions-builtin\Lora\lora.py", line 176, in load_lora module.weight.copy_(weight) RuntimeError: output with shape **[64, 320]** doesn't match the broadcast shape **[64, 320, 64, 320]**

Mine looks even more outrageous, with 2 bits missing straight away. 2.1 sd model But using the lora-add extension, it works. Hopefully it will be fixed soon, because it needs to be used with another lora weight extension, which can only be used by filling in the prompt. -Translated by deepl

I have the same problem as you, would you like to ask if you have resolved it

[catboxanon]

@KohakuBlueleaf Do you think this is related to https://github.com/KohakuBlueleaf/a1111-sd-webui-lycoris/pull/40? It might actually be a good idea to get that merged into the webui anyway after the rework to support all forms of LyCORIS networks.

[catboxanon]

Nevermind, it actually is in, didn't notice at first. https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/01997f45ba089af24b03a5f614147bb0f9d8d824/extensions-builtin/Lora/networks.py#L291-L293

1.5.0 should have fixed this, so I'm going to go ahead and close this, but feel free to open a new issue if the problem persists.