ademxapp
ademxapp copied to clipboard
itijyou
Only got 76.90% over Pascal VOC2012 val set
Hi,
I tried the Pascal VOC2012 trained model provided in the repository. However, I only got 76.90%, instead of 80.84% reported in the README.. I used the latest MXNet (v0.11.0). Do you have any idea?
Here is my log for the last three images.
2017-11-20 16:25:04,536 Host Done 1447/1449 with speed: 1.64/s 2017-11-20 16:25:04,537 Host pixel acc: 94.11%, mean acc: 82.53%, mean iou: 76.05% 2017-11-20 16:25:04,537 Host [97.46 90.25 55.71 89.61 72.90 84.55 95.42 90.71 94.92 54.97 86.99 61.03 90.37 90.42 89.59 92.72 69.21 88.79 61.88 89.36 86.20] 2017-11-20 16:25:04,537 Host [93.12 85.17 53.60 86.79 65.93 75.40 92.33 85.52 90.12 44.63 81.11 53.32 84.97 81.70 82.75 85.10 63.58 85.11 47.66 82.99 76.17] 2017-11-20 16:25:05,090 Host Done 1448/1449 with speed: 1.64/s 2017-11-20 16:25:05,091 Host pixel acc: 94.11%, mean acc: 82.53%, mean iou: 76.06% 2017-11-20 16:25:05,091 Host [97.46 90.25 55.71 89.61 72.90 84.55 95.42 90.71 94.92 54.97 86.99 61.03 90.39 90.42 89.59 92.72 69.21 88.79 61.95 89.36 86.20] 2017-11-20 16:25:05,091 Host [93.11 85.17 53.60 86.79 65.93 75.40 92.33 85.52 90.12 44.63 81.11 53.32 84.98 81.70 82.75 85.10 63.58 85.11 47.88 82.99 76.17] 2017-11-20 16:25:05,788 Host Done 1449/1449 with speed: 1.64/s 2017-11-20 16:25:05,788 Host pixel acc: 94.11%, mean acc: 82.53%, mean iou: 76.09% 2017-11-20 16:25:05,789 Host [97.46 90.25 55.71 89.61 72.90 84.62 95.42 90.71 94.92 54.97 86.99 61.03 90.39 90.42 89.59 92.72 69.21 88.79 61.95 89.36 86.20] 2017-11-20 16:25:05,789 Host [93.10 85.17 53.60 86.79 65.93 76.00 92.33 85.52 90.12 44.63 81.11 53.32 84.98 81.70 82.75 85.10 63.58 85.11 47.88 82.99 76.17] 2017-11-20 16:25:05,789 Host Done in 882.66 s.
I simply ported to python3 and tried with mxnet 1.0, then get the results as the author's.
2017-12-07 21:43:22,509 Host Done 1449/1449 with speed: 0.18/s 2017-12-07 21:43:22,509 Host pixel acc: 95.70%, mean acc: 86.26%, mean iou: 80.84% 2017-12-07 21:43:22,509 Host [98.54 95.84 86.65 93.99 85.36 87.12 95.66 92.31 97.00 54.13 90.22 60.82 92.64 92.83 93.37 93.17 76.37 91.94 62.55 88.68 82.28] 2017-12-07 21:43:22,510 Host [95.17 92.26 69.49 91.99 77.32 80.42 93.83 89.02 92.56 44.12 86.38 57.93 88.94 86.07 87.06 88.22 70.05 89.14 53.59 85.80 78.37] 2017-12-07 21:43:22,510 Host Done in 8155.81 s.
2017-12-08 00:33:29,299 Host Done 1449/1449 with speed: 0.19/s 2017-12-08 00:33:29,299 Host pixel acc: 96.22%, mean acc: 87.79%, mean iou: 82.87% 2017-12-08 00:33:29,299 Host [98.52 96.00 87.58 93.95 82.92 81.21 97.88 92.68 97.04 59.27 95.60 69.96 92.80 95.10 93.32 94.92 73.03 95.16 72.35 92.74 81.57] 2017-12-08 00:33:29,300 Host [95.66 93.92 70.93 90.37 77.40 75.07 96.51 90.65 92.08 49.72 93.36 65.84 88.20 91.67 89.10 90.44 68.30 92.45 59.21 90.17 79.18] 2017-12-08 00:33:29,300 Host Done in 7752.58 s.
maybe you would try the newest version, mxnet 1.0
I too am now trying to reproduce results. It would be really helpful if we could get version specifications for packages used to achieve the published results (e.g. a pip requirements.txt would be great). I am achieving 75-76% using:
(ademxapp) austin@vader:~/ssd/ademxapp$ pip freeze
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7.
certifi==2018.11.29
chardet==3.0.4
graphviz==0.8.1
h5py==2.9.0
idna==2.6
mxnet-cu90==1.4.0
numpy==1.13.3
Pillow==5.4.1
PyYAML==3.13
requests==2.21.0
six==1.12.0
urllib3==1.22
(ademxapp) austin@vader:~/ssd/ademxapp$ python issegm/voc.py --data-root data/VOCdevkit --output output --phase val --weights models/voc_rna-a1_cls21_s8_coco_ep-0001.params --split val --test-scales 500 --test-flipping --gpus 0
. . .
2019-03-05 16:06:11,561 Host pixel acc: 93.92%, mean acc: 87.64%, mean iou: 75.17%
2019-03-05 16:06:11,562 Host
[95.27 96.00 87.58 93.95 82.92 81.21 97.88 92.68 97.04 59.27 95.60 69.96
92.80 95.10 93.32 94.92 73.03 95.16 72.35 92.74 81.57]
2019-03-05 16:06:11,562 Host
[92.70 83.09 44.98 83.29 67.80 70.50 92.56 85.14 86.84 43.02 84.93 61.79
83.36 83.14 78.65 82.37 56.09 83.02 55.05 85.35 74.76]
2019-03-05 16:06:11,562 Host Done in 815.40 s.
. . .
. . .
(ademxapp) austin@vader:~/ssd/ademxapp$ python issegm/voc.py --data-root data/VOCdevkit --output output --phase val --weights models/voc_rna-a1_cls21_s8_ep-0001.params --split val --test-scales 500 --te
st-flipping --gpus 0
2019-03-08 14:17:06,440 Host pixel acc: 93.51%, mean acc: 86.12%, mean iou: 73.47%
2019-03-08 14:17:06,441 Host
[95.39 95.84 86.65 93.99 85.36 87.25 95.66 92.31 97.00 54.13 90.22 60.82
92.64 92.83 93.37 93.17 76.37 91.94 62.55 88.68 82.28]
2019-03-08 14:17:06,441 Host
[92.37 81.35 44.01 84.87 66.81 74.85 90.05 83.48 87.34 39.09 79.47 56.18
83.87 78.38 77.38 80.51 57.96 80.32 50.04 81.35 73.16]
2019-03-08 14:17:07,063 Host Done 1449/1449 with speed: 1.82/s
2019-03-08 14:17:07,063 Host pixel acc: 93.50%, mean acc: 86.11%, mean iou: 73.50%
2019-03-08 14:17:07,064 Host
[95.40 95.84 86.65 93.99 85.36 87.12 95.66 92.31 97.00 54.13 90.22 60.82
92.64 92.83 93.37 93.17 76.37 91.94 62.55 88.68 82.28]
2019-03-08 14:17:07,064 Host
[92.36 81.35 44.01 84.87 66.81 75.55 90.05 83.48 87.34 39.09 79.47 56.18
83.87 78.38 77.38 80.51 57.96 80.32 50.04 81.35 73.16]
2019-03-08 14:17:07,064 Host Done in 798.10 s.
I have begun trying to port the code to Python 3, but I am hoping to retrain this network on my own data, so I fear that might be complicated, but I'll plan to post an update later.
What's your training parameters?
mcever [email protected] 於 2019年3月6日 週三 上午8:19寫道：
I too am now trying to reproduce results. It would be really helpful if we could get version specifications for packages used to achieve the published results (e.g. a pip requirements.txt would be great). I am achieving 75-76% using:
(ademxapp) austin@vader:~/ssd/ademxapp$ pip freeze DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. certifi==2018.11.29 chardet==3.0.4 graphviz==0.8.1 h5py==2.9.0 idna==2.6 mxnet-cu90==1.4.0 numpy==1.13.3 Pillow==5.4.1 PyYAML==3.13 requests==2.21.0 six==1.12.0 urllib3==1.22
(ademxapp) austin@vader:~/ssd/ademxapp$ python issegm/voc.py --data-root data/VOCdevkit --output output --phase val --weights models/voc_rna-a1_cls21_s8_coco_ep-0001.params --split val --test-scales 500 --test-flipping --gpus 0 . . . 2019-03-05 16:06:11,561 Host pixel acc: 93.92%, mean acc: 87.64%, mean iou: 75.17% 2019-03-05 16:06:11,562 Host [95.27 96.00 87.58 93.95 82.92 81.21 97.88 92.68 97.04 59.27 95.60 69.96 92.80 95.10 93.32 94.92 73.03 95.16 72.35 92.74 81.57] 2019-03-05 16:06:11,562 Host [92.70 83.09 44.98 83.29 67.80 70.50 92.56 85.14 86.84 43.02 84.93 61.79 83.36 83.14 78.65 82.37 56.09 83.02 55.05 85.35 74.76] 2019-03-05 16:06:11,562 Host Done in 815.40 s.
I have begun trying to port the code to Python 3, but I am hoping to retrain this network on my own data, so I fear that might be complicated, but I'll plan to post an update later.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/itijyou/ademxapp/issues/41#issuecomment-469913476, or mute the thread https://github.com/notifications/unsubscribe-auth/AEipxO2pJ5ZOyRxf-pkCJ_wRU7xRuvn6ks5vTwmegaJpZM4QlIWj .
I have not begun training yet. I was hoping I could simply reproduce the reported results with voc_rna-a1_cls21_s8_coco_ep-0001.params before trying to train on my own data. You can see exactly the command and its output above.
I too am now trying to reproduce results. It would be really helpful if we could get version specifications for packages used to achieve the published results (e.g. a pip requirements.txt would be great). I am achieving 75-76% using:
(ademxapp) austin@vader:~/ssd/ademxapp$ pip freeze DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. certifi==2018.11.29 chardet==3.0.4 graphviz==0.8.1 h5py==2.9.0 idna==2.6 mxnet-cu90==1.4.0 numpy==1.13.3 Pillow==5.4.1 PyYAML==3.13 requests==2.21.0 six==1.12.0 urllib3==1.22(ademxapp) austin@vader:~/ssd/ademxapp$ python issegm/voc.py --data-root data/VOCdevkit --output output --phase val --weights models/voc_rna-a1_cls21_s8_coco_ep-0001.params --split val --test-scales 500 --test-flipping --gpus 0 . . . 2019-03-05 16:06:11,561 Host pixel acc: 93.92%, mean acc: 87.64%, mean iou: 75.17% 2019-03-05 16:06:11,562 Host [95.27 96.00 87.58 93.95 82.92 81.21 97.88 92.68 97.04 59.27 95.60 69.96 92.80 95.10 93.32 94.92 73.03 95.16 72.35 92.74 81.57] 2019-03-05 16:06:11,562 Host [92.70 83.09 44.98 83.29 67.80 70.50 92.56 85.14 86.84 43.02 84.93 61.79 83.36 83.14 78.65 82.37 56.09 83.02 55.05 85.35 74.76] 2019-03-05 16:06:11,562 Host Done in 815.40 s. . . . . . . (ademxapp) austin@vader:~/ssd/ademxapp$ python issegm/voc.py --data-root data/VOCdevkit --output output --phase val --weights models/voc_rna-a1_cls21_s8_ep-0001.params --split val --test-scales 500 --te st-flipping --gpus 0 2019-03-08 14:17:06,440 Host pixel acc: 93.51%, mean acc: 86.12%, mean iou: 73.47% 2019-03-08 14:17:06,441 Host [95.39 95.84 86.65 93.99 85.36 87.25 95.66 92.31 97.00 54.13 90.22 60.82 92.64 92.83 93.37 93.17 76.37 91.94 62.55 88.68 82.28] 2019-03-08 14:17:06,441 Host [92.37 81.35 44.01 84.87 66.81 74.85 90.05 83.48 87.34 39.09 79.47 56.18 83.87 78.38 77.38 80.51 57.96 80.32 50.04 81.35 73.16] 2019-03-08 14:17:07,063 Host Done 1449/1449 with speed: 1.82/s 2019-03-08 14:17:07,063 Host pixel acc: 93.50%, mean acc: 86.11%, mean iou: 73.50% 2019-03-08 14:17:07,064 Host [95.40 95.84 86.65 93.99 85.36 87.12 95.66 92.31 97.00 54.13 90.22 60.82 92.64 92.83 93.37 93.17 76.37 91.94 62.55 88.68 82.28] 2019-03-08 14:17:07,064 Host [92.36 81.35 44.01 84.87 66.81 75.55 90.05 83.48 87.34 39.09 79.47 56.18 83.87 78.38 77.38 80.51 57.96 80.32 50.04 81.35 73.16] 2019-03-08 14:17:07,064 Host Done in 798.10 s.I have begun trying to port the code to Python 3, but I am hoping to retrain this network on my own data, so I fear that might be complicated, but I'll plan to post an update later.
@mcever hello，I follow your setting for python2.7, and I test the model with voc_rna-a1_cls21_s8_coco_ep-0001.params provided by the author. I only got the miou result is 45.38%. Do you know the reason or can you give me some advice? Thanks~
