E2FGVI icon indicating copy to clipboard operation
E2FGVI copied to clipboard

Evaluation data from table 1

Open BloodLemonS opened this issue 1 year ago • 5 comments
trafficstars

How to obtain evaluation data for table 1? The values obtained when testing the DAVIS data of the release model using the provided evaluation.py file differ significantly from those in the paper.

BloodLemonS avatar Jan 17 '24 02:01 BloodLemonS

Just follow our instructions to perform the evaluation. You may need to check the installed requirements.

Paper99 avatar Feb 13 '24 15:02 Paper99

I have checked the installation requirements and found that installing different versions of third-party libraries due to differences in device hardware configuration may result in significant calculation errors?

BloodLemonS avatar Feb 18 '24 02:02 BloodLemonS

We have tested on both 3090 and V100, and the metrics are consistent. Could you please tell me whether you used our mask for evaluation?

Paper99 avatar Feb 23 '24 04:02 Paper99

Yes, we conducted the test on the 3070 graphics card using 50 provided test datasets. Except for one dataset with an inf test result, the average PSNR of the other datasets was around 20

BloodLemonS avatar Feb 23 '24 06:02 BloodLemonS

This is very weird. Please carefully check whether the environment configuration is the same as ours. The difference in GPU will not lead to different results.

Paper99 avatar Feb 23 '24 06:02 Paper99