diffusers
diffusers copied to clipboard
huggingface
multiple prediction options in ddpm, ddim
starting to work on discussion in #778. Please contribute and leave feedback. This is mostly a placeholder for my work right now as I figure out how to do it.
Some relevant repositories:
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.
Yes we indeed need this now I think :-) (also for dance diffusion)
I want to make a colab comparing the prediction for training on one scheduler to start (make sure it works). Then we need to make it on more schedulers. Though if I get DDPM done this weekend, it would be good first issues for people to add and test it on other schedulers.
I'm new to contributing and so I'm a little confused about what I should be doing. Should I clone the changes and make a colab to compare with original predictions?
Hey @Pie31415,
Since you mentioned you were interested in this PR, I think it'd be super useful to do a PR review here :-)
@Pie31415 Another really useful thing would be to just verify the implementation from the original papers and links above. This is a pretty tricky port so I will do this too, but it would be hugely useful.
For example, I actually think the DDIM implementation is much closer than DDPM.
FYI here are some examples for DDPM butterfly generations from @bglick13 , we want to see similar results on DDIM than maybe merge this initial version.
That's very interesting here actually - @patil-suraj @anton-l could you also take a look? :-)
DDIM will hopefully be ready for review soon too.
Results training on it are still a little pixelated, but you can clearly see the shape of a butterfly. I'm guessing I have something not quite right with the variance calculation. Will hopefully have updates here soon!
Update for the diffusers team (@patrickvonplaten , @anton-l , @patil-suraj ). We updated DDIM now (promising results), and I'll add tests / fix merge issues this afternoon.
@patrickvonplaten this should be go to go. Now, this leaves only DPMSolverMultistepSchedulerTest with predict_epsilon rather than prediction_type, but I think it is okay until v-prediction expands in the library.
Lots more good work from @bglick13
Added more deprecating across the board. I tried to address @patrickvonplaten's comment above, but would like a double check on that!
The code isn't as clear, but you can see some details on model parametrization in the SD 2.0 code here.
The option parametrization = "v" doesn't show up in the schedulers, though (search query here).
- noise prediction function is here
- not much in the ddim / ddpm files, which is confusing.
- I asked on twitter for clarity.
@patil-suraj @patrickvonplaten @bglick13 (see y'all after thanksgiving)
Closing this as the changes were integrated into #1505 and other earlier PRs. The added type of v-prediction for DDIM will be addressed in #1010.