[FEATURE]: Integrate StableDiffusion Dreambooth with ColossalAI

[0xdevalias]

Describe the feature

I'm aware of the current examples/integration for StableDiffusion in this repo, including it's options for finetuning:

• https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/diffusion

It would be awesome if there was an examples/code for training the Dreambooth style of finetuning/adding new concepts as well:

• https://dreambooth.github.io/
• https://github.com/XavierXiao/Dreambooth-Stable-Diffusion

  • This is an implementtaion of Google's Dreambooth with Stable Diffusion

• https://github.com/huggingface/diffusers/tree/main/examples/dreambooth
• https://github.com/ShivamShrirao/diffusers/tree/main/examples/dreambooth
  • (a fork that has various tweaks/improvements/optimisations hacked in) diff
• https://github.com/TheLastBen/fast-stable-diffusion

  • fast-stable-diffusion, +25-50% speed increase + memory efficient + DreamBooth

  • https://github.com/TheLastBen/diffusers/tree/main/examples/dreambooth
    • (a fork that has various tweaks/improvements/optimisations hacked in) diff

See also:

• https://github.com/huggingface/diffusers/issues/1212
• https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/4606
• https://github.com/d8ahazard/sd_dreambooth_extension/issues/174
• https://github.com/TheLastBen/fast-stable-diffusion/issues/493
• https://github.com/ShivamShrirao/diffusers/issues/129

[Sze-qq]

Hi, thanks for reaching out! I think it’s pretty awesome. Can I check it with our team and get back to you ASAP?

[0xdevalias]

Can I check it with our team and get back to you ASAP?

Of course! Looking forward to hearing back :)

[0xdevalias]

Can I check it with our team and get back to you ASAP?

@Sze-qq Any updates on this?

[binmakeswell]

https://github.com/hpcaitech/ColossalAI/tree/main/examples/images/dreambooth Done. Thanks.