Different image resolutions (training and sampling)

[quentinkaci]

https://github.com/lucidrains/denoising-diffusion-pytorch/blob/de378158e5e1a3fbe5a45b0da09f8644e3f1c719/denoising_diffusion_pytorch/denoising_diffusion_pytorch.py#L457

Hello,

Do you think it is possible to train a DDPM with images that have different resolutions? And therefore sample from the model some images at different resolutions?

I am currently trying it but the samples seem not good. Do you see any upgrades in the current architecture/training in order to face this issue?

Thanks for your help.

[lucidrains]

@quentinkaci hey Quentin, yea, i'm planning on open sourcing https://cascaded-diffusion.github.io/ soon in this repository

how big is your training set and how many steps did you train for?

[quentinkaci]

@lucidrains I am training on approximately 11k non-square images that have resolutions in the range [320, 512]. All the images have different resolutions. I don't crop/resize the images because I am planning to use the DDPM as a denoiser so I will need to generate images at different non-square resolutions.

Regarding https://cascaded-diffusion.github.io/, I don't think that it is well suited for my problem. I was planning to use only one DDPM that can handle images at different resolutions. Do you think it's possible?

[sgbaird]

Related:

• https://github.com/Janspiry/Image-Super-Resolution-via-Iterative-Refinement
• https://github.com/Janspiry/Palette-Image-to-Image-Diffusion-Models

Haven't tried SR3, and haven't been able to get Palette to run to completion on the example datasets, despite a lot of troubleshooting and dealing with some errors/bugs already.