ControlNetInpaintPipeline: Errors for loading Controlnet Diffusers folder

[little-misfit]

Thanks for your work, After I converted the Controlnet.pth to diffusers, an error still occurred while loading. Did my conversion fail?How can I solve this problem？The error log is as follows：

The config attributes {'controlnet_hint_channels': 3} were passed to UNet2DConditionModel, but are not expected and will be ignored. Please verify your config.json configuration file. Traceback (most recent call last): File "/home/sd/Harddisk/zxp/ControlNet-for-Diffusers/Test.py", line 8, in pipe_control = StableDiffusionControlNetInpaintPipeline.from_pretrained("./control_sd15_seg",torch_dtype=torch.float16).to('cuda') File "/home/sd/miniconda3/envs/Controlnet/lib/python3.9/site-packages/diffusers/pipelines/pipeline_utils.py", line 865, in from_pretrained loaded_sub_model = load_method(os.path.join(cached_folder, name), **loading_kwargs) File "/home/sd/miniconda3/envs/Controlnet/lib/python3.9/site-packages/diffusers/models/modeling_utils.py", line 567, in from_pretrained raise ValueError( ValueError: Cannot load <class 'diffusers.models.unet_2d_condition.UNet2DConditionModel'> from ./control_sd15_seg/controlnet because the following keys are missing: up_blocks.1.resnets.2.conv_shortcut.weight, ... .... up_blocks.2.resnets.2.norm2.bias, up_blocks.3.resnets.0.norm2.weight, up_blocks.2.resnets.0.conv1.bias, up_blocks.1.resnets.1.norm1.bias. Please make sure to pass low_cpu_mem_usage=False and device_map=None if you want to randomely initialize those weights or else make sure your checkpoint file is correct.

[haofanwang]

Can you provide your full script? Which controlnet model do you convert? @little-misfit

[little-misfit]

@haofanwang python /home/sd/Harddisk/zxp/diffusers/scripts/convert_controlnet_to_diffusers.py --checkpoint_path control_sd15_seg.pth --dump_path control_SD15_seg --device cpu

thanks

[little-misfit]

Can you provide your full script? Which controlnet model do you convert? @little-misfit

I'm trying to convert a “seg_controlnet", which is fllow your //ControlNet + Inpainting//

[haofanwang]

What is your diffusers version? Please install diffusers using the provided commit id instead of the official repo. I know that diffusers has officially supported this op, but there may have some slight difference. Once you successfully convert it, then you can load it with the recent diffusers version. @little-misfit