Relation between the metric Mean for MPII Val and [email protected] for MPII test

[abhi1kumar]

Thank you for releasing the code. The README of this repo lists out the validation accuracies of the MPII dataset in terms of Mean. On the other hand, the table 3 of the paper lists out the [email protected] numbers.

It would be great if you could tell what is the relationship between these two metrics? If they are different, would you mind placing the [email protected] validation numbers for the pose_resnet_152 and pose_hrnet_w32.

Thanks again.

[abhi1kumar]

I just found out the answer to my question. Both Mean and [email protected] seem to be very closely related.

If we look into the file mpii.py, the Mean is indeed the average of the PCKh of points [0,1,2,3,4,5,8,9,10,11,12,13,14,15]. Note that point 6 and 7 are not used for calculating the mean since the mask of the masked array is set to true for these two points.

Please refer lines 161-165 of the file mpii.py

PCKh = np.ma.array(PCKh, mask=False)
PCKh.mask[6:8] = True

jnt_count = np.ma.array(jnt_count, mask=False)
jnt_count.mask[6:8] = True

It would be really great if you could tell the reason for masking out these points and not using them in Mean [email protected] calculation. Thanks again

[iskaiwen]

I just found out the answer to my question. Both Mean and [email protected] seem to be very closely related.

If we look into the file mpii.py, the Mean is indeed the average of the PCKh of points [0,1,2,3,4,5,8,9,10,11,12,13,14,15]. Note that point 6 and 7 are not used for calculating the mean since the mask of the masked array is set to true for these two points.

Please refer lines 161-165 of the file mpii.py

PCKh = np.ma.array(PCKh, mask=False)
PCKh.mask[6:8] = True

jnt_count = np.ma.array(jnt_count, mask=False)
jnt_count.mask[6:8] = True

It would be really great if you could tell the reason for masking out these points and not using them in Mean [email protected] calculation. Thanks again

The PCKh performance metric is the percentage of joints with predicted locations that are no further than half of the head segment length from the ground truth.

"PCKh total" excludes the pelvis and thorax joints from the calculation, presumably because they are very easy to predict given that the approximate person center is provided.