diffusers icon indicating copy to clipboard operation
diffusers copied to clipboard

Support optionally using karras sigmas in the k_diffusion wrapper pipeline

Open apolinario opened this issue 2 years ago • 4 comments

Is your feature request related to a problem? Please describe. K-Diffusion enables users to generate sigmas with k_diffusion.sampling.get_sigmas_karras. diffusers currently do not support using those to generate the sigmas with its k_diffusion wrapper

Describe the solution you'd like Enable users to optionally generate karras sigmas similarly to how AUTO1111 does it https://github.com/AUTOMATIC1111/stable-diffusion-webui/blob/44c46f0ed395967cd3830dd481a2db759fda5b3b/modules/sd_samplers.py#L453

apolinario avatar Dec 09 '22 13:12 apolinario

I will try to allocate some time in the next weeks to look into it (can't promise to get it done soon though). Just wondering: do we have some results/benchmarks to know whether karras sigmas work better ?

patrickvonplaten avatar Dec 13 '22 09:12 patrickvonplaten

I don't think they necessarily work better.

It just works differently and the community enjoys having the possibility of having this variations for artistic purposes when using tools that support it such as AUTO1111 and any other based on k_diffusion so I think it would be great for diffusers to also enable such variations. But I can't say it works better (some people believe so subjectively)

apolinario avatar Dec 13 '22 09:12 apolinario

In case one is interested in taking a look? @williamberman @yiyixuxu @pcuenca @patil-suraj

patrickvonplaten avatar Jan 22 '23 21:01 patrickvonplaten

Here a solution that should work well: https://github.com/huggingface/diffusers/issues/1887#issuecomment-1399263545 . Hope to soon find time to implement it :-)

patrickvonplaten avatar Jan 23 '23 07:01 patrickvonplaten

这里有一个应该运行良好的解决方案:#1887(评论)。希望尽快找到时间来实施它:-)

Did you add the karras sigmas to diffusers's scheduler? I want to use DPM++ 2M karras but I only find DPMSolver multi-step.

ym-ctw avatar Mar 13 '23 09:03 ym-ctw

Not yet @ym-ctw - we'd be more than happy to review a PR though :-)

patrickvonplaten avatar Mar 13 '23 16:03 patrickvonplaten

Not yet @ym-ctw - we'd be more than happy to review a PR though :-)

Thanks, I really hope you could add karras especially in DPM++ series soon. Because now much more new model are using 'DPM++ 2M karras' to get much better results than 'eular a' and others schedulers, most of these models are from 'civitai.com.' Hope you could keep an emphasis on it, Thanks for your effort.

ym-ctw avatar Mar 16 '23 09:03 ym-ctw