Cannot convert SD to diffusers

[brurpo]

Describe the bug

Trying to convert a ckpt fine tuned from dreambooth (2.1 768) to diffusers yield the following error: KeyError: 'cond_stage_model.model.text_projection' This happens either locally or using a space from hugging face: https://huggingface.co/spaces/diffusers/sd-to-diffusers

Reproduction

You can ask for access to the repo I am trying to convert at https://huggingface.co/brurpo/sbtbakeoff

Logs

E:\sd\diffusers030\diffusers-0.11.1\diffusers-0.11.1\scripts\convert_original_stable_diffusion_to_diffusers.py  --checkpoint_path "E:\sd\output_gui\sbtbakeoff\model.ckpt" --original_config_file "E:\sd\output_gui\v2-inference-v.yaml"  --image_size 768  --prediction_type v-prediction --upcast_attn True --dump_path "E:\sd\output_gui\sbtbakeoff_sd"
NOTE: Redirects are currently not supported in Windows or MacOs.
A matching Triton is not available, some optimizations will not be enabled.
Error caught was: No module named 'triton'
global_step key not found in model
Traceback (most recent call last):
  File "E:\sd\diffusers030\diffusers-0.11.1\diffusers-0.11.1\scripts\convert_original_stable_diffusion_to_diffusers.py", line 982, in <module>
    text_model = convert_open_clip_checkpoint(checkpoint)
  File "E:\sd\diffusers030\diffusers-0.11.1\diffusers-0.11.1\scripts\convert_original_stable_diffusion_to_diffusers.py", line 751, in convert_open_clip_checkpoint
    d_model = int(checkpoint["cond_stage_model.model.text_projection"].shape[0])
KeyError: 'cond_stage_model.model.text_projection'

System Info

diffusers 0.11.1, sd 2.1 768, python 3.9 locally or huggingface space

[camenduru]

maybe this works https://huggingface.co/spaces/camenduru/converter

[tweeter0830]

Also running into this bug

[camenduru]

If your model is public, please send me the link. I want to try converting it

[tweeter0830]

It isn't public, but it was generated via convert_diffusers_to_original_stable_diffusion.

I changed line 753 to read:

    if "cond_stage_model.model.text_projection" in checkpoint:
        d_model = int(checkpoint["cond_stage_model.model.text_projection"].shape[0])
    else:
        d_model = 1024

My guess is that something is not being propagated properly through the "convert_diffusers_to_original_stable_diffusion.py" script. Mu solution is just a dirty hack.

[brurpo]

maybe this works https://huggingface.co/spaces/camenduru/converter Thanks! I did try this one too, it doesnt work. Not sure if its something from dreambooth or a bug with the script itself.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[rvorias]

same trying to convert sd 21 512 checkpoint to diffusers, this key is not in the original inference.yaml config either

[boloniusMonk]

I have been getting this error trying to convert https://civitai.com/models/8124/a-to-zovya-rpg-artists-tools-15-and-21 SD 2.1 768 V1. Using the change listed above, I was able to convert the SD 2.1 model to ONNX.

I changed line 753 to read:

    if "cond_stage_model.model.text_projection" in checkpoint:
        d_model = int(checkpoint["cond_stage_model.model.text_projection"].shape[0])
    else:
        d_model = 1024

[patrickvonplaten]

PR is merged, should be solved! Thanks a lot for the help ☺️