EfficientNet-PyTorch icon indicating copy to clipboard operation
EfficientNet-PyTorch copied to clipboard

Published 20 hours ago • verified publisher icon lukemelas

Pretrained weights don't give same accuracy as stated in repo

Open neiladit opened this issue 5 years ago • 6 comments

I tried using pretrained weights for efficientnet-b0 and efficientnet-b1 and evaluated it on Imagenet but did not get the same Accuracy (rank 1 and rank 5) that is mentioned in the table here. I'm using the evaluation code from the repository itself. Here is what I got:

efficientnet-b0: Prec@1 74.802 Prec@5 91.998 efficientnet-b1: Prec@1 74.638 Prec@5 91.674

However, some of them gave similar results to the README table: efficientnet-b2: Prec@1 79.408 Prec@5 94.604 efficientnet-b3: Prec@1 81.196 Prec@5 95.530

Can you verify this and let me know if there is an issue?

neiladit avatar Sep 15 '20 02:09 neiladit

hi mate EfficientNet is very sensitive to resolution if you have seen the paper, you can see that what they are optimizing is compound of depth width and resolution of the network. Therefore, you MUST upscale the given image to specified image scale

I've been there, haha and got same accuracy as yours

ooodragon94 avatar Sep 17 '20 14:09 ooodragon94

@ooodragon94 Thanks for the comment. Doesn't this line deal with that?

neiladit avatar Sep 17 '20 15:09 neiladit

hm, that seems legit maybe you are using different loader?? or maybe this repo has been changed

sorry I'm not sure with that

ooodragon94 avatar Sep 17 '20 15:09 ooodragon94

@ooodragon94 Since I'm using the file from the repository, I don't think anything is different. Were you able to replicate the results?

neiladit avatar Sep 17 '20 16:09 neiladit

I was It was quite long ago though; I manually had to upscale images to get the specified results

ooodragon94 avatar Sep 17 '20 16:09 ooodragon94

@lukemelas can you confirm if there is a modification required in the Imagenet evaluation code? Thanks a lot!

neiladit avatar Sep 18 '20 04:09 neiladit