stable-diffusion.cpp icon indicating copy to clipboard operation
stable-diffusion.cpp copied to clipboard
Published 2 weeks ago • verified publisher icon leejet

Control net doesn't work with Sd1.5

Open wandbrandon opened this issue 11 months ago • 7 comments 

[INFO ] control.hpp:434  - loading control net from '../models/controlnet/sd15openpose11.safetensors'
[DEBUG] ggml_extend.hpp:1111 - control_net params backend buffer size =  937.60 MB(RAM) (340 tensors)
[INFO ] model.cpp:888  - load ../models/controlnet/sd15openpose11.safetensors using safetensors format
[DEBUG] model.cpp:959  - init from '../models/controlnet/sd15openpose11.safetensors'
[DEBUG] model.cpp:1698 - loading tensors from ../models/controlnet/sd15openpose11.safetensors
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.0.bias | f16 | 1 [16, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.0.weight | f16 | 4 [3, 3, 16, 16, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.1.bias | f16 | 1 [32, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.1.weight | f16 | 4 [3, 3, 16, 32, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.2.bias | f16 | 1 [32, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.2.weight | f16 | 4 [3, 3, 32, 32, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.3.bias | f16 | 1 [96, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.3.weight | f16 | 4 [3, 3, 32, 96, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.4.bias | f16 | 1 [96, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.4.weight | f16 | 4 [3, 3, 96, 96, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.5.bias | f16 | 1 [256, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.blocks.5.weight | f16 | 4 [3, 3, 96, 256, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.conv_in.bias | f16 | 1 [16, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.conv_in.weight | f16 | 4 [3, 3, 3, 16, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.conv_out.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_cond_embedding.conv_out.weight | f16 | 4 [3, 3, 256, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.0.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.0.weight | f16 | 4 [1, 1, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.1.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.1.weight | f16 | 4 [1, 1, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.10.bias | f16 | 1 [1280, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.10.weight | f16 | 4 [1, 1, 1280, 1280, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.11.bias | f16 | 1 [1280, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.11.weight | f16 | 4 [1, 1, 1280, 1280, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.2.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.2.weight | f16 | 4 [1, 1, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.3.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.3.weight | f16 | 4 [1, 1, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.4.bias | f16 | 1 [640, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.4.weight | f16 | 4 [1, 1, 640, 640, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.5.bias | f16 | 1 [640, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.5.weight | f16 | 4 [1, 1, 640, 640, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.6.bias | f16 | 1 [640, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.6.weight | f16 | 4 [1, 1, 640, 640, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.7.bias | f16 | 1 [1280, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.7.weight | f16 | 4 [1, 1, 1280, 1280, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.8.bias | f16 | 1 [1280, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.8.weight | f16 | 4 [1, 1, 1280, 1280, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.9.bias | f16 | 1 [1280, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_down_blocks.9.weight | f16 | 4 [1, 1, 1280, 1280, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_mid_block.bias | f16 | 1 [1280, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'controlnet_mid_block.weight | f16 | 4 [1, 1, 1280, 1280, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'conv_in.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'conv_in.weight | f16 | 4 [3, 3, 4, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.norm.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.norm.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.proj_in.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.proj_in.weight | f16 | 4 [1, 1, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.proj_out.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.proj_out.weight | f16 | 4 [1, 1, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn1.to_k.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn1.to_out.0.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn1.to_out.0.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn1.to_q.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn1.to_v.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn2.to_k.weight | f16 | 2 [768, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn2.to_out.0.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn2.to_out.0.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn2.to_q.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.attn2.to_v.weight | f16 | 2 [768, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.ff.net.0.proj.bias | f16 | 1 [2560, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.ff.net.0.proj.weight | f16 | 2 [320, 2560, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.ff.net.2.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.ff.net.2.weight | f16 | 2 [1280, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.norm1.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.norm1.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.norm2.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.norm2.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.norm3.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.0.transformer_blocks.0.norm3.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.norm.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.norm.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.proj_in.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.proj_in.weight | f16 | 4 [1, 1, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.proj_out.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.proj_out.weight | f16 | 4 [1, 1, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn1.to_k.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn1.to_out.0.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn1.to_out.0.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn1.to_q.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn1.to_v.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn2.to_k.weight | f16 | 2 [768, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn2.to_out.0.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn2.to_out.0.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn2.to_q.weight | f16 | 2 [320, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.attn2.to_v.weight | f16 | 2 [768, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.ff.net.0.proj.bias | f16 | 1 [2560, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.ff.net.0.proj.weight | f16 | 2 [320, 2560, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.ff.net.2.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.ff.net.2.weight | f16 | 2 [1280, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.norm1.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.norm1.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.norm2.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.norm2.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.norm3.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.attentions.1.transformer_blocks.0.norm3.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.downsamplers.0.conv.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.downsamplers.0.conv.weight | f16 | 4 [3, 3, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.conv1.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.conv1.weight | f16 | 4 [3, 3, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.conv2.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.conv2.weight | f16 | 4 [3, 3, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.norm1.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.norm1.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.norm2.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.norm2.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.time_emb_proj.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.0.time_emb_proj.weight | f16 | 2 [1280, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.conv1.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.conv1.weight | f16 | 4 [3, 3, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.conv2.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.conv2.weight | f16 | 4 [3, 3, 320, 320, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.norm1.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.norm1.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.norm2.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.norm2.weight | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.time_emb_proj.bias | f16 | 1 [320, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.0.resnets.1.time_emb_proj.weight | f16 | 2 [1280, 320, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.norm.bias | f16 | 1 [640, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.norm.weight | f16 | 1 [640, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.proj_in.bias | f16 | 1 [640, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.proj_in.weight | f16 | 4 [1, 1, 640, 640, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.proj_out.bias | f16 | 1 [640, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.proj_out.weight | f16 | 4 [1, 1, 640, 640, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_k.weight | f16 | 2 [640, 640, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_out.0.bias | f16 | 1 [640, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_out.0.weight | f16 | 2 [640, 640, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_q.weight | f16 | 2 [640, 640, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn1.to_v.weight | f16 | 2 [640, 640, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_k.weight | f16 | 2 [768, 640, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_out.0.bias | f16 | 1 [640, 1, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_out.0.weight | f16 | 2 [640, 640, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_q.weight | f16 | 2 [640, 640, 1, 1, 1]' in model file
[INFO ] model.cpp:1868 - unknown tensor 'down_blocks.1.attentions.0.transformer_blocks.0.attn2.to_v.weight | f16 | 2 [768, 640, 1, 1, 1]' in model file
...
[ERROR] control.hpp:449  - load control net tensors from model loader failed
new_sd_ctx_t failed

Getting this error when trying to use control net.

Using this control net with a sd1.5 model: https://huggingface.co/lllyasviel/control_v11p_sd15_openpose/tree/main

Here is my command:

bin/sd 
-m ../models/mistoonAnime_v30.safetensors 
--lora-model-dir ../models/loras 
--taesd ../models/taesd1.safetensors 
--steps 3 -v 
-p "masterpiece, best quality, 1girl, angel, blue eyes, bobcut, blonde hair<lora:lcm-lora-sdv1-5:1>" 
-H 256 
-W 384 
--cfg-scale 1 
--strength 1.00 
--sampling-method lcm 
--control-net ../models/controlnet/openposefp16.safetensors 
--control-image ../models/controlnet/openposetest.png

wandbrandon avatar Feb 04 '25 00:02 wandbrandon

Interesting. I tried the safetensors model from the link, really not works. (As i remember it worked some months ago). But the pickletensors model file works well: https://huggingface.co/lllyasviel/ControlNet-v1-1/blob/main/control_v11p_sd15_openpose.pth

fszontagh avatar Feb 04 '25 07:02 fszontagh

Looks like the tensor naming convention is completely diferent. I'm guessing diffusers requires different tensor names for control Maybe this one would work? https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/blob/main/control_v11p_sd15_openpose_fp16.safetensors I can't test it right now.

stduhpf avatar Feb 04 '25 12:02 stduhpf

Looks like the tensor naming convention is completely diferent. I'm guessing diffusers requires different tensor names for control Maybe this one would work? https://huggingface.co/comfyanonymous/ControlNet-v1-1_fp16_safetensors/blob/main/control_v11p_sd15_openpose_fp16.safetensors I can't test it right now.

This model is works.

fszontagh avatar Feb 04 '25 12:02 fszontagh

Yes it seems it only works with the models from that repo. Is there planned support for the other control net models?

wandbrandon avatar Feb 08 '25 19:02 wandbrandon

Yes it seems it only works with the models from that repo. Is there planned support for the other control net models?

I looked into it quickly, it seems harder than just mapping the block names, the whole structure is slightly different. I don't think it would be worth the effort. It would probably be best to just put a link to repos with the supported controlnets in the readme instead.

stduhpf avatar Feb 08 '25 20:02 stduhpf

Do we support control nets for other models as well besides 1.5? What about this?

https://huggingface.co/xinsir/controlnet-union-sdxl-1.0/tree/main

wandbrandon avatar Feb 17 '25 19:02 wandbrandon

Do we support control nets for other models as well besides 1.5? What about this?

https://huggingface.co/xinsir/controlnet-union-sdxl-1.0/tree/main

Yeah, I tried the CN model you linked, as well as two other CN models that work in Forge. However, I encountered numerous "tensor ... not in model file" errors, causing the run to fail.

As far as I know, SDXL CN is not supported in sd.cpp. In Forge, SDXL CN does work, but the results are worse compared to SD 1.5 ControlNet. From what I’ve seen, CN performs best with SD 1.5.

fszontagh avatar Feb 18 '25 12:02 fszontagh