ControlNet Is it possible to use with Stable Diffusion XL?

Is it possible to use it as it is? Is a specific support required?

Jul 10 '23 04:07 notdanilo

Any updates on this? Plan / Roadmap @lllyasviel

Aug 02 '23 17:08 Michaelvirga

use HuggingFace Diffuser's training script https://github.com/huggingface/diffusers/blob/main/examples/controlnet/README_sdxl.md

Sep 17 '23 07:09 geroldmeisinger

https://huggingface.co/stabilityai/control-lora

Sep 17 '23 12:09 geroldmeisinger

Anyone working on this for the ControlNet repo? I guess we would need to change the config so that tool_add_control.py works. And has someone tried to train it on the link https://github.com/lllyasviel/ControlNet/issues/461#issuecomment-1722415410 ?

Apr 10 '24 15:04 jbu5105

I have managed to create a ControlNet checkpoint for Stable Diffusion XL (sd_xl). It is not a beautiful implementation. It seems that the UnetModel in ldm/modules/diffusionmodules/openaimodel.py is outdated to use the config from sd_xl. The sd_xl implementation can be found in https://github.com/Stability-AI/generative-models/blob/main/sgm/modules/diffusionmodules/openaimodel.py

I am not sure if anyone is intereseted in this. Maybe this repository is not being used anymore and people are using more diffusers or generative-models

Apr 12 '24 12:04 jbu5105