ml-stable-diffusion
ml-stable-diffusion copied to clipboard
apple
SDXL Turbo support?
Not sure what is required, but I thought I'd add an issue for discussion? I would love to integrate this into my project. Could be a huge leap forward.
https://huggingface.co/stabilityai/sdxl-turbo
Asked the same in Moshi Diffusion https://github.com/godly-devotion/MochiDiffusion/issues/348
Turbo just works to my knowledge, i've tried it using the comfyui node and it works and that projects just pulling in this project for the backend
Turbo just works to my knowledge, i've tried it using the comfyui node and it works and that projects just pulling in this project for the backend
I converted it myself, and could get image generation working by removing the refiner (which I guess is optional), but saw no improvement in number of steps required to generate an image, so I don't think it just works. I asked in the MochiDiffusion discord and was told: "SDXLTurbo needs a turbo-specific noise scheduler and an Euler ancestral sampler to work, neither is available at the moment with ml-stable-diffusion"
I am way more a front-end coder, and don't understand the inner workings of the SD models, so not really sure how to interpret that quote, but maybe someone else can comment?
We have a working implementation over at Odyssey. Essentially, what you need to do is to grab the LCMScheduler class from GuernikaKit (massive kudos to them, by the way), massage it a bit to fit the diffusion pipeline class found on this repo, set the timeStepSpacing to trailing, and then use a guidance scale of exactly 1 to run a Turbo model. Feel free to hit me up if anyone needs more detailed guidance.
I don't want to spam here, but in case anyone else wants to implement SDXL Turbo support in his/her app, all you need to do it's add SDE or LCM schedulers support and you're ready to go. I implemented it in my own app (DreamFusion, completely free), if you interested feel free to try it out!
PS: DreamFusion also support v-predict model (Terminus XL Gamma v2 beta1), turbo models (2.1 turbo, DreamShaperXL turbo), inpaint, instruct and other things. I basically made it for my self because of lacks some funcs I needed it ml-stable-difusion around 4 month ago but decided to share it with others and it's free!
