Got very low ACC in MPII valid set and COCO valid set

[ustcjinggg]

I follow the instruction by README,yet get a very low AP ,i tried many times in different GPU,still got 69AP in MPII val set and 11.5AP in COCO val set. detail in the following:

root@db8a72eec293:/home/pb12000407/test/HR-Net# python tools/test.py \

--cfg experiments/mpii/hrnet/w32_256x256_adam_lr1e-3.yaml \
TEST.MODEL_FILE models/pytorch/pose_mpii/pose_hrnet_w32_256x256.pth

......

=> loading model from models/pytorch/pose_mpii/pose_hrnet_w32_256x256.pth => load 2958 samples Test: [0/47] Time 32.100 (32.100) Loss 0.0198 (0.0198) Accuracy 0.698 (0.698)

Arch	Head	Shoulder	Elbow	Wrist	Hip	Knee	Ankle	Mean	[email protected]
pose_hrnet	72.988	71.977	67.922	64.502	67.094	66.049	63.132	67.963	27.916
#################################################
root@f23979639adf:/home/pb12000407/test/HRnet# python tools/test.py --cfg experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml TEST.MODEL_FILE models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth TEST.USE_GT_BBOX False
=> creating output/coco/pose_hrnet/w32_256x192_adam_lr1e-3
=> creating log/coco/pose_hrnet/w32_256x192_adam_lr1e-3_2019-06-18-01-03
Namespace(cfg='experiments/coco/hrnet/w32_256x192_adam_lr1e-3.yaml', dataDir='', logDir='', modelDir='', opts=['TEST.MODEL_FILE', 'models/pytorch/pose_coco/pose_hrnet_w32_256x192.pth', 'TEST.USE_GT_BBOX', 'False'], prevModelDir='')
......

Test: [800/814] Time 2.434 (2.829) Loss 0.0002 (0.2789) Accuracy 0.059 (0.004) Test: [810/814] Time 2.541 (2.829) Loss 0.0001 (0.2778) Accuracy 0.000 (0.004) => writing results json to output/coco/pose_hrnet/w32_256x192_adam_lr1e-3/results/keypoints_val2017_results_0.json Loading and preparing results... DONE (t=5.17s) creating index... index created! Running per image evaluation... Evaluate annotation type keypoints DONE (t=13.20s). Accumulating evaluation results... DONE (t=0.51s). Average Precision (AP) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.155 Average Precision (AP) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.202 Average Precision (AP) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.173 Average Precision (AP) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.222 Average Precision (AP) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.194 Average Recall (AR) @[ IoU=0.50:0.95 | area= all | maxDets= 20 ] = 0.587 Average Recall (AR) @[ IoU=0.50 | area= all | maxDets= 20 ] = 0.691 Average Recall (AR) @[ IoU=0.75 | area= all | maxDets= 20 ] = 0.635 Average Recall (AR) @[ IoU=0.50:0.95 | area=medium | maxDets= 20 ] = 0.524 Average Recall (AR) @[ IoU=0.50:0.95 | area= large | maxDets= 20 ] = 0.675

Arch	AP	Ap .5	AP .75	AP (M)	AP (L)	AR	AR .5	AR .75	AR (M)	AR (L)
pose_hrnet	0.155	0.202	0.173	0.222	0.194	0.587	0.691	0.635	0.524	0.675
#########################################################
Is there a problem with the model the author offer? or I need to train first? Please help ,MaydayMayday!..

[leoxiaobin]

Could you provide the version of pytorch you are using? And which GPU are you using?

[ustcjinggg]

Thanks for your replying ,the version of pytorch is 1.1.0 and the GPU is Tesla K80,plus the python version is 3.6.8

[MRRRKING]

I may have the same problem as you. When I train using COCO dataset, I can get correct results. But when I test, the loss and the accuracy are very low. Did you solve the problem?

Test: [0/3254] Time 4.684 (4.684) Loss 0.0000 (0.0000) Accuracy 0.000 (0.000) Test: [100/3254] Time 0.623 (0.694) Loss 0.0001 (0.0001) Accuracy 0.000 (0.006) Test: [200/3254] Time 0.612 (0.657) Loss 0.0000 (0.0001) Accuracy 0.000 (0.006) Test: [300/3254] Time 0.604 (0.644) Loss 0.0000 (0.0001) Accuracy 0.000 (0.005) Test: [400/3254] Time 0.666 (0.638) Loss 0.0000 (0.0001) Accuracy 0.000 (0.004) Test: [500/3254] Time 0.565 (0.634) Loss 0.0002 (0.0001) Accuracy 0.000 (0.003) Test: [600/3254] Time 0.594 (0.632) Loss 0.0000 (0.0001) Accuracy 0.000 (0.003) Test: [700/3254] Time 0.631 (0.630) Loss 0.0004 (0.0001) Accuracy 0.000 (0.002)

[Vincent-Hoo]

I may have the same problem as you. When I train using COCO dataset, I can get correct results. But when I test, the loss and the accuracy are very low. Did you solve the problem?

Test: [0/3254] Time 4.684 (4.684) Loss 0.0000 (0.0000) Accuracy 0.000 (0.000) Test: [100/3254] Time 0.623 (0.694) Loss 0.0001 (0.0001) Accuracy 0.000 (0.006) Test: [200/3254] Time 0.612 (0.657) Loss 0.0000 (0.0001) Accuracy 0.000 (0.006) Test: [300/3254] Time 0.604 (0.644) Loss 0.0000 (0.0001) Accuracy 0.000 (0.005) Test: [400/3254] Time 0.666 (0.638) Loss 0.0000 (0.0001) Accuracy 0.000 (0.004) Test: [500/3254] Time 0.565 (0.634) Loss 0.0002 (0.0001) Accuracy 0.000 (0.003) Test: [600/3254] Time 0.594 (0.632) Loss 0.0000 (0.0001) Accuracy 0.000 (0.003) Test: [700/3254] Time 0.631 (0.630) Loss 0.0004 (0.0001) Accuracy 0.000 (0.002)

I have the same problem, zero accuracy when testing the coco validation set. Did you solve the problem?

[yanbiaogit]

I may have the same problem as you. When I train using COCO dataset, I can get correct results. But when I test, the loss and the accuracy are very low. Did you solve the problem? Test: [0/3254] Time 4.684 (4.684) Loss 0.0000 (0.0000) Accuracy 0.000 (0.000) Test: [100/3254] Time 0.623 (0.694) Loss 0.0001 (0.0001) Accuracy 0.000 (0.006) Test: [200/3254] Time 0.612 (0.657) Loss 0.0000 (0.0001) Accuracy 0.000 (0.006) Test: [300/3254] Time 0.604 (0.644) Loss 0.0000 (0.0001) Accuracy 0.000 (0.005) Test: [400/3254] Time 0.666 (0.638) Loss 0.0000 (0.0001) Accuracy 0.000 (0.004) Test: [500/3254] Time 0.565 (0.634) Loss 0.0002 (0.0001) Accuracy 0.000 (0.003) Test: [600/3254] Time 0.594 (0.632) Loss 0.0000 (0.0001) Accuracy 0.000 (0.003) Test: [700/3254] Time 0.631 (0.630) Loss 0.0004 (0.0001) Accuracy 0.000 (0.002)

I have the same problem, zero accuracy when testing the coco validation set. Did you solve the problem?

请问您当时解决这个问题了吗？我现在测试的结果也是全为0