diffusers
diffusers copied to clipboard
[Community] Is there a code for pre-training stable diffusion?
This project works great. Is there a code for pre-training stable diffusion?
Hey @980202006,
Thanks for the nice words, we currently don't have a pretraining script for stable diffusion. Currently I don't think we have the time to support such a new script, but we would welcome any contribution from the community.
Also cc'ing @patil-suraj here FYI.
Thank you!
The train_text_to_image
script can be adapted for pre-training stable diffusion. With SD, pre-training and fine-tuning objectives are similar. https://github.com/huggingface/diffusers/tree/main/examples/text_to_image, so to train from scratch we'll have to load random models, initialize the correct scheduler, handle data pre-processing etc.
@patil-suraj Thank you! Do you know the approximate loss of vae convergence? The vae loss I trained is very large.
No, I haven't trained VAE. Also, stable diffusion does not train VAE, it uses pre-trained VAE checkpoints trained with taming-transformer. In stable diffusion only the unet
is trained, the rest is frozen.