ERRNet icon indicating copy to clipboard operation
ERRNet copied to clipboard

About the resize problem of real20

Open Rachel-kk opened this issue 2 years ago • 6 comments

Hi,I've resized the short edge of images in real20 dataset to 512(used torchvision.transforms.Resize).. I tested your released model but it got psnr of 21.89, ssim of 0.804.

I don't understand why the psnr I tested is not the same as your paper(the psnr of 22.89, the ssim of 0.803 in your paper) , I hope to get your answer.

Rachel-kk avatar Sep 03 '22 09:09 Rachel-kk

Hi, I just uploaded my testing data into GoogleDrive from the local computer. Could you please check whether you can reproduce our results

Vandermode avatar Sep 04 '22 01:09 Vandermode

Hi, @Vandermode Thank you for providing your testing data. I downloaded it and retested on your released model, the result is as follows:

Real(20) Postcard(179) Wild(52) Soilobject(200)
your paper 22.89 / 0.803 22.04 / 0.876 24.25 / 0.853 24.87 / 0.896
retest results 22.91 / 0.806 22.07 / 0.877 24.21 / 0.857 24.85 / 0.898

I observed that the psnr error is within 0.03, and it's right. Next, I will retrain your released model and reproduce your results.

Rachel-kk avatar Sep 04 '22 05:09 Rachel-kk

Hi, I run your code on GitHub. However, I didn't reproduce the same performance as the paper. For VOC2012 dataset, following the way you said, I crop the center region with size 224 x 224. Is there anywhere else that would affect performance?

Rachel-kk avatar Sep 05 '22 12:09 Rachel-kk

Could you please tell me more information about your environment? for example, the PyTorch version, the GPU version, etc..

Vandermode avatar Sep 05 '22 13:09 Vandermode

Note this work was done many years ago, and actually, I tried to reproduce its results several months ago, but I found a very weird issue---the reproduction was successful in old GPU architectures (e.g., 1080Ti, V100), but often failed in SoTA GPUs (e.g., 3090Ti, A100)... I don't figure out how this could be possible, maybe you encountered the same issue

Vandermode avatar Sep 05 '22 13:09 Vandermode

Hi, I train on two environments separately, as follow. the Pytorch version is 1.7.1+cuda10.1; the GPU version is TITAN V. the Pytorch version is 1.10.1+cuda11.1; the GPU version is RTX 2080

Rachel-kk avatar Sep 06 '22 11:09 Rachel-kk