Crashed, when I run the ControlNet model on iPhone 14

[jiangdi0924]

Thank you to all the contributors for their contributions that have allowed me to experience such an interesting package. I have developed an application that can run ml-stable-diffusion smoothly on iPhone14. However, I encountered some issues when trying to incorporate ControlNet features.

Problem description

When I run ControlNet on an actual iPhone 14 to generate images, it crashes due to excessive memory usage (it works fine on a simulator). Is there might be any incorrect settings when I converted the model?

python -m python_coreml_stable_diffusion.torch2coreml --convert-controlnet lllyasviel/sd-controlnet-scribble --unet-support-controlnet --convert-unet --convert-text-encoder --convert-vae-decoder --convert-vae-encoder -o sdv1_5_chunked_controlnet --model-version runwayml/stable-diffusion-v1-5 --bundle-resources-for-swift-cli --chunk-unet --attention-implementation SPLIT_EINSUM

Model

[SaladDays831]

Hi @jiangdi0924 :) Were you able to resolve this? If so - what model/conversion did you use?

[jiangdi0924]

Hi @jiangdi0924 :) Were you able to resolve this? If so - what model/conversion did you use?

Not yet 🥶， I will test the new features (quantize-nbits) of ml-stable-difusion 1.0.0

[SaladDays831]

Ahhh, was hoping that you found a solution 😁 If you have any updates - pls ping!

We also have some discussions here that might help you