Question about the correspondence between SH and H3.6M joint positions.

[hacchi-kh]

I have a question about the correspondence between SH and H3.6M joint positions as defined in this program.

The program ignores the "Neck/Nose" joints in the H3.6M joint data in order to make the 2D data of SH and H3.6M compatible. I think there is a problem with this "Neck/Nose" ignoring, and I think the joint that should be ignored on H3.6M is "Spine".

To prove it, I used pyplot to visualize the H3.6M data loaded in "load_data" and the SH data loaded in "load_stacked_hourglass". Here are the results. (The top and bottom of the image are reversed due to pyplot's display settings)

Human3.6M

Stacked Hourglass

Comparing these two diagrams, the joint that exist in the H3.6M data but not in the SH data is considered to be "Spine". The "Neck/Nose" joint is present in both data. However, in the program, "Spine" exists in both H3.6M and SH data.

(Here is a list of H3.6M and SH joints defined in data_utils.py)

# Joints in H3.6M -- data has 32 joints, but only 17 that move; these are the indices. H36M_NAMES = ['']*32 H36M_NAMES[0] = 'Hip' H36M_NAMES[1] = 'RHip' H36M_NAMES[2] = 'RKnee' H36M_NAMES[3] = 'RFoot' H36M_NAMES[6] = 'LHip' H36M_NAMES[7] = 'LKnee' H36M_NAMES[8] = 'LFoot' H36M_NAMES[12] = 'Spine' H36M_NAMES[13] = 'Thorax' H36M_NAMES[14] = 'Neck/Nose' H36M_NAMES[15] = 'Head' H36M_NAMES[17] = 'LShoulder' H36M_NAMES[18] = 'LElbow' H36M_NAMES[19] = 'LWrist' H36M_NAMES[25] = 'RShoulder' H36M_NAMES[26] = 'RElbow' H36M_NAMES[27] = 'RWrist'

# Stacked Hourglass produces 16 joints. These are the names. SH_NAMES = ['']*16 SH_NAMES[0] = 'RFoot' SH_NAMES[1] = 'RKnee' SH_NAMES[2] = 'RHip' SH_NAMES[3] = 'LHip' SH_NAMES[4] = 'LKnee' SH_NAMES[5] = 'LFoot' SH_NAMES[6] = 'Hip' SH_NAMES[7] = 'Spine' SH_NAMES[8] = 'Thorax' SH_NAMES[9] = 'Head' SH_NAMES[10] = 'RWrist' SH_NAMES[11] = 'RElbow' SH_NAMES[12] = 'RShoulder' SH_NAMES[13] = 'LShoulder' SH_NAMES[14] = 'LElbow' SH_NAMES[15] = 'LWrist'

The current program ignores "Neck/Nose" of H3.6M joints in order to be compatible with SH. However, I think that what should be ignored in H3.6M joints is "Spine".

Please let me know if there's anything wrong with my opinion.

And I'm not an English speaker, so I use a translation tool. Please let me know if you don't understand any part of my English.

[una-dinosauria]

Hi @hacchi-kh,

This is a great find! It might explain why I was never able to reproduce certain numbers in the paper (issue https://github.com/una-dinosauria/3d-pose-baseline/issues/60) -- cc @matteorr.

Thanks for reporting this.