[Bug]: Some LoRAs do not load when using sd-webui-forge

[Vektor8298]

Checklist

• [X] The issue exists after disabling all extensions
• [X] The issue exists on a clean installation of webui
• [ ] The issue is caused by an extension, but I believe it is caused by a bug in the webui
• [X] The issue exists in the current version of the webui
• [ ] The issue has not been reported before recently
• [ ] The issue has been reported before but has not been fixed yet

What happened?

When trying to load a specific LoRA (Add More Details - Detail Enhancer / Tweaker), it does not get loaded, the console does not show the LoRA being loaded nor the final image reflects it.

Steps to reproduce the problem

1. Add the magic word to enable the LoRA
2. Generate an image
3. Check results to see the LoRA was not applied

What should have happened?

LoRA should be loaded and applied to the generation.

What browsers do you use to access the UI ?

Mozilla Firefox, Google Chrome, Other

Sysinfo

sysinfo-2024-02-18-02-04.json

Console logs

`glibc version is 2.39
Check TCMalloc: libtcmalloc_minimal.so.4
libtcmalloc_minimal.so.4 is linked with libc.so,execute LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so.4
Python 3.10.6 (main, Feb  9 2024, 20:12:38) [GCC 13.2.1 20230801]
Version: f0.0.14v1.8.0rc-latest-184-g43c9e3b5
Commit hash: 43c9e3b5ce1642073c7a9684e36b45489eeb4a49
Launching Web UI with arguments: --listen --always-offload-from-vram --attention-quad
amdgpu.ids: No such file or directory
Total VRAM 8176 MB, total RAM 15873 MB
Set vram state to: NORMAL_VRAM
Always offload VRAM
Device: cuda:0 AMD Radeon Graphics : native
VAE dtype: torch.float32
Using sub quadratic optimization for cross attention, if you have memory or speed issues try using: --attention-split
ControlNet preprocessor location: /mnt/ts512/stable-diffusion-webui-forge/models/ControlNetPreprocessor
Loading weights [386a466799] from /mnt/ts512/stable-diffusion-webui-forge/models/Stable-diffusion/based69_v10.safetensors
model_type EPS
UNet ADM Dimension 0
Using split attention in VAE
Working with z of shape (1, 4, 32, 32) = 4096 dimensions.
Using split attention in VAE
extra {'cond_stage_model.clip_l.text_projection', 'cond_stage_model.clip_l.logit_scale'}
Couldn't find VAE named WD1-4-kl-f8-anime2-bless09.safetensors; using None instead
To load target model SD1ClipModel
Begin to load 1 model
Moving model(s) has taken 0.15 seconds
Model loaded in 6.0s (load weights from disk: 2.6s, forge load real models: 2.7s, calculate empty prompt: 0.7s).
[nltk_data] Downloading package wordnet to /home/vektor/nltk_data...
[nltk_data]   Package wordnet is already up-to-date!
[nltk_data] Downloading package omw-1.4 to /home/vektor/nltk_data...
[nltk_data]   Package omw-1.4 is already up-to-date!
Tag Autocomplete: Could not locate model-keyword extension, Lora trigger word completion will be limited to those added through the extra networks menu.
[-] ADetailer initialized. version: 24.1.2, num models: 14
2024-02-17 23:19:50,192 - ControlNet - INFO - ControlNet UI callback registered.
Running on local URL:  http://0.0.0.0:7860

To create a public link, set `share=True` in `launch()`.
Startup time: 25.7s (prepare environment: 4.7s, import torch: 4.2s, import gradio: 1.0s, setup paths: 1.3s, other imports: 0.8s, load scripts: 11.7s, create ui: 1.6s, gradio launch: 0.3s).
To load target model SD1ClipModel
Begin to load 1 model
unload clone 0
Moving model(s) has taken 0.21 seconds
To load target model BaseModel
Begin to load 1 model
Moving model(s) has taken 0.55 seconds
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:19<00:00,  1.57it/s]
To load target model AutoencoderKL███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:18<00:00,  1.61it/s]
Begin to load 1 model
Moving model(s) has taken 0.38 seconds

0: 640x448 1 face, 10.3ms
Speed: 9.3ms preprocess, 10.3ms inference, 77.7ms postprocess per image at shape (1, 3, 640, 448)
To load target model SD1ClipModel
Begin to load 1 model
Moving model(s) has taken 0.16 seconds
To load target model BaseModel
Begin to load 1 model
Moving model(s) has taken 0.40 seconds
100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 13/13 [00:08<00:00,  1.58it/s]
To load target model AutoencoderKL
Begin to load 1 model
Moving model(s) has taken 0.25 seconds
Total progress: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 30/30 [00:29<00:00,  1.01it/s]
Loading VAE weights specified in settings: /mnt/ts512/stable-diffusion-webui-forge/models/VAE/WD1-4-kl-f8-anime2-bless09.safetensors█████████████████████████████████████████████████| 30/30 [00:29<00:00,  1.61it/s]
VAE weights loaded.`

Additional information

GPU Driver updated recently, Arch Linux, Python 3.10.6

[rerere14]

same problem

[magejosh]

Same problem here. It's strange though, seems like some Lora do not work at all, some just destroy the generation, and some work as expected. Problem is they all work as expected in comfyui and in a1111.

[IMT-AI]

I am also facing the same issue... some LORAs will be applied, and some won't. I still cannot figure out what makes the LORAs be reflected on the generated image and what not...

[KNX-PT-6]

So far, have you checked the Lora button in WebUI to see if your Lora is showing up there? After some investigation, it seems that all the .safetensors Lora cannot be found by the Lora picker, and I am guessing that's what makes some Lora not working. .ckpt Loras can still be found and are shown in the picker; I assume those are probably those Lora that work?

[Nerogante]

My Loras were not being detected when setting set A1111_HOME=C:\SD-1.6 But they appeared after changing \ to / set A1111_HOME=C:/SD-1.6

[wardensc2]

I have same problem, remove those Addetail Lora, everything work fine

[KNX-PT-6]

So far, have you checked the Lora button in WebUI to see if your Lora is showing up there? After some investigation, it seems that all the .safetensors Lora cannot be found by the Lora picker, and I am guessing that's what makes some Lora not working. .ckpt Loras can still be found and are shown in the picker; I assume those are probably those Lora that work?

cough um, it seems that it's just those Loras, in my case, are not compatible with XL models. I switched back to 1.5 models and they all appeared once again. I have no clue why that's the case, but my problem was solved. yay...?

[dvelm]

I have same problem, remove those Addetail Lora, everything work fine

What should be removed?

[vamzii]

if the lora isnt available in the models directory and is still called from prompt, it doesnt show that the lora isnt available as shown in the official automatic1111 release, it generates image which is totally out of line

[IMT-AI]

My issue is intermittent... Sometimes it gets applied, sometimes not... I imagine a command must "block" the application of the loras if it is ran before... but I still cannot figure it out.

[dvelm]

Fooocus read and use the lora in the folder but this not

[IMT-AI]

Just updating, today I found ONE of the reasons. I used PNG Info to get the prompt from an image and tried generating. It went until a LORA I didn't have and did not use the rest. So I removed the non-existing LORA from the prompt and the rest loaded correctly. It seems it is not the only thing that makes Forge "forget' to apply a LORA, but one of them.

[cokimax2]

I have the same issue. In my case, some lora and many of textual inversions are not listed neither.

[Sarimae]

still a problem.

[pioniere]

The problem is still occurring.

[ikun-404]

So far, have you checked the Lora button in WebUI to see if your Lora is showing up there? After some investigation, it seems that all the .safetensors Lora cannot be found by the Lora picker, and I am guessing that's what makes some Lora not working. .ckpt Loras can still be found and are shown in the picker; I assume those are probably those Lora that work?

cough um, it seems that it's just those Loras, in my case, are not compatible with XL models. I switched back to 1.5 models and they all appeared once again. I have no clue why that's the case, but my problem was solved. yay...?

Is it really Perhaps the mechanism is like this