mmagic
mmagic copied to clipboard
BUG in DeepFillv2 (or other inpainting model): eval results is different for different test batch size
I am using DeepFillv2 model for image inpainting. But I found that eval results is different for different test batch size.
In img_inpainting_dataset.py
, the final eval result is calculated in function evaluate
, here you use sum
for batch eval results. But in batch eval, the reduction maybe mean
, so that the eval results is different for different test batch size.
I further reviewed the code and found that psnr
and ssim
are calculated completely different when batch_size > 1
from it is when batch_size = 1
.
Hi @zml24 Would you mind providing more information for us to reproduce this issue? For example, the config file you used, the script you run.
Closing due to inactivity, please reopen if there are any further problems.