VQGAN-CLIP icon indicating copy to clipboard operation
VQGAN-CLIP copied to clipboard

Sample probability

Open abrichr opened this issue 3 years ago • 1 comments

Given a new image, we would like to determine the probability of an image given the trained model parameters (e.g. as described in http://proceedings.mlr.press/v97/balaji19a/balaji19a.pdf).

Creating a new cog endpoint in predict.py is an obvious starting point. Any suggestions on where to go from there would be greatly appreciated!

A surrogate approach might be to simply sample from the discriminator. If the discriminator estimates the probability that a sample came from the data distribution rather than model distribution, and for a particular image that probability is low, then the image has low probability given the training data. Assuming the GAN has been trained to generate images from the training data, does it follow that the image has low probability given the model parameters?

So instead of determining the probability of an image given the trained model parameters, we are determining the probability of the image given the training data. Is that right?

abrichr avatar Jan 26 '22 17:01 abrichr

Similarly, how can we use the text that was used to generate an image to calculate the image's probability?

abrichr avatar Jan 29 '22 05:01 abrichr