t2i adapters seem not working

[iqddd]

Loading model: t2iadapter_style-fp16 [0e2e8330]
Loaded state_dict from [/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/models/t2iadapter_style-fp16.safetensors]
Error running process: /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py
Traceback (most recent call last):
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/modules/scripts.py", line 409, in process
    script.process(p, *script_args)
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 742, in process
    model_net = self.load_control_model(p, unet, model, lowvram)
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 541, in load_control_model
    model_net = self.build_control_model(p, unet, model, lowvram)
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/controlnet.py", line 579, in build_control_model
    network = network_module(
  File "/content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/scripts/adapter.py", line 81, in __init__
    self.control_model.load_state_dict(state_dict)
  File "/usr/local/lib/python3.9/dist-packages/torch/nn/modules/module.py", line 1671, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Adapter:
	Missing key(s) in state_dict: "body.0.block1.weight", "body.0.block1.bias", "body.0.block2.weight", "body.0.block2.bias", "body.1.block1.weight", "body.1.block1.bias", "body.1.block2.weight", "body.1.block2.bias", "body.2.in_conv.weight", "body.2.in_conv.bias", "body.2.block1.weight", "body.2.block1.bias", "body.2.block2.weight", "body.2.block2.bias", "body.3.block1.weight", "body.3.block1.bias", "body.3.block2.weight", "body.3.block2.bias", "body.4.in_conv.weight", "body.4.in_conv.bias", "body.4.block1.weight", "body.4.block1.bias", "body.4.block2.weight", "body.4.block2.bias", "body.5.block1.weight", "body.5.block1.bias", "body.5.block2.weight", "body.5.block2.bias", "body.6.block1.weight", "body.6.block1.bias", "body.6.block2.weight", "body.6.block2.bias", "body.7.block1.weight", "body.7.block1.bias", "body.7.block2.weight", "body.7.block2.bias", "conv_in.weight", "conv_in.bias". 
	Unexpected key(s) in state_dict: "ln_post.bias", "ln_post.weight", "ln_pre.bias", "ln_pre.weight", "proj", "style_embedding", "transformer_layes.0.attn.in_proj_bias", "transformer_layes.0.attn.in_proj_weight", "transformer_layes.0.attn.out_proj.bias", "transformer_layes.0.attn.out_proj.weight", "transformer_layes.0.ln_1.bias", "transformer_layes.0.ln_1.weight", "transformer_layes.0.ln_2.bias", "transformer_layes.0.ln_2.weight", "transformer_layes.0.mlp.c_fc.bias", "transformer_layes.0.mlp.c_fc.weight", "transformer_layes.0.mlp.c_proj.bias", "transformer_layes.0.mlp.c_proj.weight", "transformer_layes.1.attn.in_proj_bias", "transformer_layes.1.attn.in_proj_weight", "transformer_layes.1.attn.out_proj.bias", "transformer_layes.1.attn.out_proj.weight", "transformer_layes.1.ln_1.bias", "transformer_layes.1.ln_1.weight", "transformer_layes.1.ln_2.bias", "transformer_layes.1.ln_2.weight", "transformer_layes.1.mlp.c_fc.bias", "transformer_layes.1.mlp.c_fc.weight", "transformer_layes.1.mlp.c_proj.bias", "transformer_layes.1.mlp.c_proj.weight", "transformer_layes.2.attn.in_proj_bias", "transformer_layes.2.attn.in_proj_weight", "transformer_layes.2.attn.out_proj.bias", "transformer_layes.2.attn.out_proj.weight", "transformer_layes.2.ln_1.bias", "transformer_layes.2.ln_1.weight", "transformer_layes.2.ln_2.bias", "transformer_layes.2.ln_2.weight", "transformer_layes.2.mlp.c_fc.bias", "transformer_layes.2.mlp.c_fc.weight", "transformer_layes.2.mlp.c_proj.bias", "transformer_layes.2.mlp.c_proj.weight". 

When attempting to apply any t2i model.

[pcrii]

There are three yaml files that end in _sd14v1 if you change that portion to -fp16 it should work. I've used style and color they both work but I haven't tried keypose

sd\stable-diffusion-webui\extensions\sd-webui-controlnet\models

[iqddd]

Thanks a lot.

[arqli]

The same problem:

Loading model: t2iadapter_style_sd14v1 [202e85cc]
Loaded state_dict from [C:\stable-diffusion-webui\models\ControlNet\t2iadapter_style_sd14v1.pth]
Error running process: C:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py
Traceback (most recent call last):
  File "C:\stable-diffusion-webui\modules\scripts.py", line 386, in process
    script.process(p, *script_args)
  File "C:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 742, in process
    model_net = self.load_control_model(p, unet, model, lowvram)
  File "C:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 541, in load_control_model
    model_net = self.build_control_model(p, unet, model, lowvram)
  File "C:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\controlnet.py", line 579, in build_control_model
    network = network_module(
  File "C:\stable-diffusion-webui\extensions\sd-webui-controlnet\scripts\adapter.py", line 81, in __init__
    self.control_model.load_state_dict(state_dict)
  File "C:\stable-diffusion-webui\python\lib\site-packages\torch\nn\modules\module.py", line 1671, in load_state_dict
    raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Adapter:
        Missing key(s) in state_dict: "body.0.block1.weight", "body.0.block1.bias", "body.0.block2.weight", "body.0.block2.bias", "body.1.block1.weight", "body.1.block1.bias", "body.1.block2.weight", "body.1.block2.bias", "body.2.in_conv.weight", "body.2.in_conv.bias", "body.2.block1.weight", ....
<following omitted>

I used t2iadapter_style_sd14v1 [202e85cc] with yaml file ended with '_sd14v1', and also tried t2iadapter_style-fp16 [0e2e8330] when change the end of yaml file to '-fp16', but it didn't work either....

[pcrii]

this colab provides a .safetensor for this file your output shows "ControlNet\t2iadapter_style_sd14v1.pth" should be t2iadapter_style-fp16.safetensor

im not sure what you renamed but you can just add a cell and run theses lines after the controlnet section. it only needs done once

!cp /content/gdrive/MyDrive/sd/stable-diffusion-webui/models/t2iadapter_style_sd14v1.yaml /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/models/t2iadapter_style-fp16.yaml
!cp /content/gdrive/MyDrive/sd/stable-diffusion-webui/models/t2iadapter_keypose_sd14v1.yaml /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/models/t2iadapter_keypose-fp16.yaml
!cp /content/gdrive/MyDrive/sd/stable-diffusion-webui/models/t2iadapter_color_sd14v1.yaml /content/gdrive/MyDrive/sd/stable-diffusion-webui/extensions/sd-webui-controlnet/models/t2iadapter_color-fp16.yaml 

[TheLastBen]

@pcrii you're sure that's the cause ? if so, I'll fix it

[pcrii]

yeah its definitely a working fix ive played with style and color alot but i dont know if it would be better to rename the .safetensor to match the yaml instead

doing it this way was better for me because i only added three small files and the colab wont fetch a second instance of the model next run

https://github.com/Mikubill/sd-webui-controlnet/tree/main/models https://raw.githubusercontent.com/TheLastBen/fast-stable-diffusion/main/AUTOMATIC1111_files/CN_models.txt

[TheLastBen]

thanks, I'll try to find the right way to implement the fix

[pcrii]

this is kinda what inspired the file name change. and fixed style/color for sure. i just adjusted for the .safetensors you used. im not super confident about renaming* the keypose file but ive been doing it too

*coping with a new name. i didnt remove any of the originals

[pcrii]

"Copy corresponding config file and rename it to the same name as the model" after rereading Mikubill literally says to rename the yaml he provides.

so im 100% confident now keypose included

didnt do much testing because style and color were all i wanted but it think the t2i canny and such worked fine without needing any adjustment

[TheLastBen]

the fix should be ready in a few this should fix it https://github.com/TheLastBen/fast-stable-diffusion/commit/81a7a900fa2c0402c62a014dd2811a2722b55d6f

[arqli]

I got it.... My yaml files were in extension-controlnet-models path but .pt/.safetensor in models-controlnet path.... Put them together and everything fine....