consistency_models icon indicating copy to clipboard operation
consistency_models copied to clipboard

Is the sampling reversible? (Image to Noise)

Open discordance opened this issue 1 year ago • 3 comments

Hello! Your work is truly remarkable. I've had the opportunity to experiment with standard DDPM models using DDIM sampling, and despite being relatively slow, they exhibit a fascinating reversibility property. This allows denoising, as well as the reverse process, recreating the exact same input noise (or close).

I am seeing that there is two samplers being used here: sample_onestep and stochastic_iterative_sampler.

stochastic_iterative_sampler adds stochastic noise between each steps so is obviously not reversible. Not sure about sample_onestep.

I would be grateful if you could provide some insights into whether this reversibility property is preserved in Consistency models sampling. The examples provided do not explicitly demonstrate this aspect, and I am eager to learn more about it. Thank you for your time and expertise.

discordance avatar Apr 17 '23 16:04 discordance

I'm equally interested in this issue. Have you implemented the reverse process? I'm also curious why this issue was closed?

Tranquil1ty avatar May 09 '23 14:05 Tranquil1ty

Nope, no success here so far. Basically, what I am trying to achieve is two fold:

  • Interpolation between two images using the model.
  • Take an image and generate infinite variations around.

It's obvious how to do that once you can get from Image -> Noise because all those operation can be performed into the gaussian space and denoised again. But from what I understand, those models are learning to denoise directly, they are not anymore a score function that can be reversed at each iteration.

I have tried (at sampling time):

  • Take noise X_t, denoise it into x0, interp x0 with the other image into x1 and go one iteration more into the model with x1. The results are not excellent.

Edit: I closed the issue because I thought my question was premature but actually many people are looking for similar thing

discordance avatar May 12 '23 11:05 discordance

Nope, no success here so far. Basically, what I am trying to achieve is two fold:

  • Interpolation between two images using the model.
  • Take an image and generate infinite variations around.

It's obvious how to do that once you can get from Image -> Noise because all those operation can be performed into the gaussian space and denoised again. But from what I understand, those models are learning to denoise directly, they are not anymore a score function that can be reversed at each iteration.

I have tried (at sampling time):

  • Take noise X_t, denoise it into x0, interp x0 with the other image into x1 and go one iteration more into the model with x1. The results are not excellent.

Edit: I closed the issue because I thought my question was premature but actually many people are looking for similar thing

Hi, i'm looking for this problem's solution, are you or someone else have any idea? cause you say many people are looking for it, where should i find some place where i can discuss it?

krNeko9t avatar Dec 13 '23 08:12 krNeko9t