objectnet-template-pytorch icon indicating copy to clipboard operation
objectnet-template-pytorch copied to clipboard

Accuracy for Imagenet Overlap Only

Open JayRGopal opened this issue 1 year ago • 0 comments

Hi there,

I believe validate_and_score.py filters out the model predictions for objectnet images that are not in imagenet categories to get the number of correct predictions. However, this is divided by the total number of images in objectnet.

https://github.com/abarbu/objectnet-template-pytorch/blob/master/validate_and_score.py#L34

This code therefore gives accuracy well below the numbers reported in the paper. Modifying the total to only represent the images in categories that overlap between the two datasets fixes this error for me.

Could you please take a look and let me know if I'm missing something?

Thanks!

JayRGopal avatar Apr 28 '23 23:04 JayRGopal