mmagic icon indicating copy to clipboard operation
mmagic copied to clipboard

BUG in DeepFillv2 (or other inpainting model): eval results is different for different test batch size

Open zml24 opened this issue 2 years ago • 1 comments

I am using DeepFillv2 model for image inpainting. But I found that eval results is different for different test batch size. In img_inpainting_dataset.py, the final eval result is calculated in function evaluate, here you use sum for batch eval results. But in batch eval, the reduction maybe mean, so that the eval results is different for different test batch size.

zml24 avatar Jul 27 '22 03:07 zml24

I further reviewed the code and found that psnr and ssim are calculated completely different when batch_size > 1 from it is when batch_size = 1.

zml24 avatar Jul 27 '22 04:07 zml24

Hi @zml24 Would you mind providing more information for us to reproduce this issue? For example, the config file you used, the script you run.

zengyh1900 avatar Oct 10 '22 06:10 zengyh1900

Closing due to inactivity, please reopen if there are any further problems.

zengyh1900 avatar Oct 25 '22 05:10 zengyh1900