How to reproduce the results in the paper

[csguoh]

Hi, sorry to bother you.

In the released code of smaple_lora_int_model.py, is seems only one image is sampled as a toy example. How to reproduce the results of 500K samples and save them as npz format, to reproduce results reported in the paper?

If it is convenient to share the related code? Thank you very much !

[csguoh]

I also have another question, why the DDIM of ImageNet uses 20 timesetps while the code released seems uses more timestep 100? Does this modification influence the performance?

[ThisisBillhe]

Hi, you may refer to another issue to reproduce the results. Also, the steps of diffusion models can be adjusted as you wish.

[csguoh]

I will try it, thanks for your reply!

[csguoh]

Hi, authors!

I follow the instruction in README, and use the released code with 100 timesteps using DDIM on ImageNet, and use the guided-diffusion codebase for evaluate. However, I still can not reproduce the results in Table2. Since the 50000 smapling is too long, I use 30000 instead. The results of W4A4 (the default setup in the code) of my reproduce is as follows:

I also use the pre-trained ckpt (20 DDIM steps) you have provided and the results are as follows:

The results reported in the paper is 250.90 || 6.17 || 7.75.

I suspect the above failure of reproduce might be caused by the sample generation process. I write the sampling code by myself following the smaple_lora_inimodel. But it seems not work well for reproduce. Could you give me some help about this? Thx!