ddim icon indicating copy to clipboard operation
ddim copied to clipboard

Published 20 hours ago • verified publisher icon ermongroup

got wrong FID using pretrained model

Open ztyang196 opened this issue 2 years ago • 3 comments

Hi, I sampled 50000 images using the provided pretrained CelebA model, but got a FID of 5.83 in the setting of timesteps=1000, eta=0, which is a margin away from 3.51 reported in the paper (when timesteps is set to 100, the FID is 10.13 while 6.53 reported in the paper). May I know where the problem is? Is the total CelebA dataset used for calculating FID?

ztyang196 avatar Jun 02 '22 02:06 ztyang196

@ztyang196 Hi, I sampled 50k images just the same way as you did, however, I got a FID of around 50. I examined the synthesized images which look similar to the images shown in the paper, and I have no idea whether I calculated the FID the wrong way. Could you please share your method to calculate FID score? Thanks!

thuxmf avatar Jun 20 '22 10:06 thuxmf

Hi, I suspect that you didn’t downsample the training dataset to the same resolution as the synthesized images or you didn’t use the same way as this code to preprocess the training dataset

ztyang196 avatar Jun 20 '22 11:06 ztyang196

Thanks for your suggestions! FYI, I use pytorch-fid to calculate the fid and get better results than that reported in the paper.

thuxmf avatar Jun 20 '22 13:06 thuxmf