Evaluation Problem

[heitorrapela]

Hello there, I'm trying to evaluate some models but it is giving 0.0 in acc. I'm using Python 3.6.5 and pytorch 0.4.0

I got the validation dataset from this wget:

# Validation images (all tasks), 6.3GB 
wget -b http://www.image-net.org/challenges/LSVRC/2012/nnoupb/ILSVRC2012_img_val.tar

[Joeywzr]

Hi, have you solved it?

[heitorrapela]

No

[Cadene]

Hi @heitorrapela and @Joeywzr,

It works for me with these versions:

>>> import torch
>>> import pretrainedmodels
>>> torch.__version__
'1.0.1.post2'
>>> pretrainedmodels.__version__
'0.7.4'

CUDA_VISIBLE_DEVICES=1 python examples/imagenet_eval.py --data /local/common-data/imagenet_2012/images -a inceptionv4 -b 1 -e
=> creating model 'inceptionv4'
=> using pre-trained parameters 'imagenet'
Images transformed from size 342 to [3, 299, 299]
Test: [0/50000] Time 2.069 (2.069)      Loss 6.6667 (6.6667)    Acc@1 0.000 (0.000)     Acc@5 0.000 (0.000)
Test: [10/50000]        Time 0.052 (0.242)      Loss 8.1854 (1.5328)    Acc@1 0.000 (72.727)    Acc@5 0.000 (81.818)
Test: [20/50000]        Time 0.050 (0.152)      Loss 0.0953 (0.8392)    Acc@1 100.000 (85.714)  Acc@5 100.000 (90.476)
Test: [30/50000]        Time 0.056 (0.121)      Loss 0.0820 (0.5941)    Acc@1 100.000 (90.323)  Acc@5 100.000 (93.548)
Test: [40/50000]        Time 0.051 (0.105)      Loss 0.0740 (0.5733)    Acc@1 100.000 (87.805)  Acc@5 100.000 (95.122)
Test: [50/50000]        Time 0.053 (0.095)      Loss 0.5860 (0.4843)    Acc@1 100.000 (90.196)  Acc@5 100.000 (96.078)
Test: [60/50000]        Time 0.050 (0.088)      Loss 0.0135 (0.4207)    Acc@1 100.000 (91.803)  Acc@5 100.000 (96.721)
Test: [70/50000]        Time 0.059 (0.083)      Loss 0.0655 (0.3776)    Acc@1 100.000 (92.958)  Acc@5 100.000 (97.183)
Test: [80/50000]        Time 0.053 (0.080)      Loss 0.0857 (0.3388)    Acc@1 100.000 (93.827)  Acc@5 100.000 (97.531)
Test: [90/50000]        Time 0.053 (0.077)      Loss 0.0354 (0.3625)    Acc@1 100.000 (93.407)  Acc@5 100.000 (97.802)
Test: [100/50000]       Time 0.053 (0.075)      Loss 0.0860 (0.3696)    Acc@1 100.000 (93.069)  Acc@5 100.000 (98.020)

What about you?

[Joeywzr]

Hi @Cadene ,thanks for your reply! Could you show us your dataset? I think that's the problem. Thanks!

[Cadene]

@Joeywzr You can download it here: http://data.lip6.fr/cadene/imagenet/val.tar.gz (I should probably give easy access to it :/)

Please tell me if it works for you :)

[Joeywzr]

@Cadene Thanks a lot! That's exactly what I need!