denoising-diffusion-pytorch

denoising-diffusion-pytorch copied to clipboard

Is it possible to use this for conditional generation?

@abdalazizrashid This was transcribed from the original TensorFlow implementation to Pytorch and it does not support conditional generation.

Take a look at the this paper, they introduced a condition encoding network which you can implement, based on the code here by adding a few modifications to some functions as well as the loss to accomodate for this new condition.

Hope it helps.

I think that adding the conditional input is quite easy. You could have a look to section 5 of this paper: https://arxiv.org/pdf/2102.09672.pdf

If anyone tries this out, would be great if you could link to some code or provide a MWE. Or if anyone is aware of other implementations

Has anyone been able to apply this code for tasks like inpainting or super-resolution? They are both forms of conditional generation, and it's not clear to me how one should go about doing it.

See also https://github.com/lucidrains/denoising-diffusion-pytorch/issues/21#issuecomment-1143989193

Class-conditional generation can be found in https://github.com/openai/improved-diffusion.

I'm curious if there are any updates on this topic. I've been reviewing the SR3 and Palette papers, and I plan to test an approach of concatenating the source image with $x_{t}$ along the channel dimension to see if it works. I'm not completely sure what the source image should be, however. If we have pairs of training data in the form of (low resolution, high resolution), should we add noise to the high resolution image during the forward process and condition it on the low resolution image during reverse diffusion? The Palette paper mentions that they upsampled the low resolution source image to match the size of the high resolution image before concatenating it with the input from the forward diffusion along the channel dimension.