huggingface
Can you change the stable-diffusion Swift Package to use StableDiffusionControlNetPipeline instead of StableDiffusionPipeline?
The stable-diffusion Swift package uses the 'StableDiffusionPipeline' class. How can I change the pipeline to use the 'StableDiffusionControlNetPipeline'? I will be using the diffusion model runwaylml/stable-diffusion-v1-5 so I don't think I need to go through the step of converting the stable-diffusion-v1-5 model into a CoreML model.
Hi @yerx! You won't need to convert the stable-diffusion-v1-5 model, but you'd need to convert the ControlNet model(s) you'll be using, and then modify the pipeline code to use both the ControlNet model and the base one.
May I ask what use-case you have in mind, and what ControlNet model are you planning to use?
Hi @pcuenca , thanks for responding. I'm interested in using the Canny and Scribble ControlNet models.
I'm really excited for this feature to be made available on the Mac! Is there a process to get ControlNet to (finally!) work for yerx, myself and anyone else on the Mac? <3 Thank you so much!
I was overly excited to see something else on the SD space than on the Web UI, so that brought me here. Upon more digging, I found something shockingly good... "DrawThings:" https://apps.apple.com/us/app/draw-things-ai-generation/id6444050820
DT does everything that I'm watching on YouTube about SD. <3
I hope this addendum here to my comment I made yesterday clears my thoughts up, sorry for any confusion. Cheers and have a good one!
In light of the release of the controlnet models on HF by the CoreML team: https://huggingface.co/coreml/ControlNet-Models-For-Core-ML
Will official support for the controlnet pipeline be considered? Mochi diffusion has already implemented it if an example is needed.
