Why DEX_YCB .json annotations are different from the original dataset's .npz file

[karta2155802]

I'm working on the two datasets. I found that mano_param : pose parameters in json file are different from the original dataset's pose parameters. So I wonder what makes them different? And how did you generate the pose parameters in the annotation json file?

[namepllet]

The pose_m parameter in original dataset's annotation file is PCA coefficients.

I changed it to axis angle representation.

[karta2155802]

Could you provide the script for changing PCA coefficiets to axis angle representation? Thank you verty mush.

[namepllet]

Here are core lines we used.

from manopth.manolayer import ManoLayer
manolayer_left = ManoLayer(mano_root=osp.join(cfg.mano_path, 'mano', 'models'), flat_hand_mean=False, use_pca=True, side='left', ncomps=45)
manolayer_right = ManoLayer(mano_root=osp.join(cfg.mano_path, 'mano', 'models'), flat_hand_mean=False, use_pca=True, side='right', ncomps=45)

mano_pose = label['pose_m'][0,:48] # PCA
th_mano_pose = torch.FloatTensor(mano_pose).view(1,-1)
selected_comps = manolayer_left.th_selected_comps if hand_type == 'left' else manolayer_right.th_selected_comps
th_mano_pose[:,3:] = th_mano_pose[:,3:].mm(selected_comps)
mano_pose = th_mano_pose.numpy()[0] # Axis-angle

[karta2155802]

Hi, I have the same question on HO3D dataset. The original dataset said their hand pose annotations are already axis-angle representation and can be directly fed into mano model, but your annotation in json file are still different from theirs. But when I tried to generate hand mesh using manolayer. Results that used your hand pose annotation are more close to the real image than the original dataset's annotation. I don't get it.

[namepllet]

For the original dataset, set flat_hand_mean=True in here https://github.com/namepllet/HandOccNet/blob/65c00af06ad81f568a6ad0ff4919f7f1d6c65e44/common/utils/mano.py#L37

[karta2155802]

Thanks for the quick reply. So what's the reson that you chosed to use average hand coefficients instead of flat hand? Does it make the performance better? Cause I read some other papers still use flat hand coefficients as groundtruth. And how to generate the average hand coefficients from the original dataset?

[namepllet]

Just for convenience. I think the performance will not much different since the average hand pose is constant offset.

The average hand pose is not for original dataset(HO3D). It is saved in MANO_RIGHT.pkl file.

[heartStrive]

In the HO3D v3 dataset, I simply write the pose parameter in the pickle to a json file, and the 21 joints are stored in the json file in the way you mentioned. However, the json file generated in this way is poorly used to train the model, could you please share the code for handling the pose parameter? Thank you very much!