pretrained model & model trained from scratch

[dong-x16]

Thanks for releasing the training code. I have some questions about the pretrained model. I trained my own model on coco train2017 from scratch with one GPU(batchsize = 4), and I evaluated my own model on data/coco_pose/valid_id, and got a mAP of 0.592 for single scale evaluation. When I evaluated on val2017, I got a mAP of about 0.53 for single scale, while the pretrained model got a mAP of about 0.625 for single scale. What's more, when evaluated on test_dev2017, my own model got about 0.522 for single scale, and the pretrained model got about 0.565 for single scale.

I don't kown why the mAP significantly different on val2017 and test_dev2017, did the pretrained model trained on coco train2017+val2017 except the 500 images in data/coco_pose/valid_id ? or maybe because of not using multiple GPUs ?

[panfengli]

is it generally to first train on train_2014, test on val_2014 to get the optimized parameters, then test on test_2014? I think it is the year 2014, not 2017. By the way, what do you put in data_dir = 'coco/images' since ann_path = 'coco/annotations/person_keypoints_train2014.json'

[dong-x16]

I slightly channged the training data from coco train2014 in the code to coco train2017 in order to including more training data, the order is first training on train2017 from scratch, then test on val2017 and test_dev2017 for single scale with refine.

[hellojialee]

@dong-x16 How do you evaluate on 2017 test dataset?

[hellojialee]

@dong-x16 Could you please share your results on mscoco 2017 test-dev subset without refinement? I need to compare the difference. Thank you!

[coordxyz]

@dong-x16 Could you please share your training details, such as training and validation loss, total iterations, input/output size, etc? Thanks a lot.