How to convert 3dkeypoint from relative to absolute values

[ghost]

I tried on "body3d_two_stage_video_demo.py" and expoted the value of variable "pose_lift_results_vis" to a json file. But "keypoint_3d" key seems to be relative coordinates.

The config/checkpoint file I used is shown below. det_config = 'mmpose/demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py' det_checkpoint = 'https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth' pose_detector_config = 'mmpose/configs/body/2d_kpt_sview_rgb_img/topdown_heatmap/coco/hrnet_w48_coco_256x192.py' pose_detector_checkpoint = 'https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_256x192-b9e0b3ab_20200708.pth' pose_lifter_config = 'mmpose/configs/body/3d_kpt_sview_rgb_vid/video_pose_lift/h36m/videopose3d_h36m_243frames_fullconv_supervised_cpn_ft.py' pose_lifter_checkpoint = 'https://download.openmmlab.com/mmpose/body3d/videopose/videopose_h36m_243frames_fullconv_supervised_cpn_ft-88f5abbb_20210527.pth'

A part of the output json file is shown below.

{"track_id": 0, "keypoints": [[478.2337341308594, 968.0533447265625, 0.8321810364723206], [436.89129638671875, 968.0533447265625, 0.831661581993103], [459.44171142578125, 1140.93994140625, 0.9109681248664856], [436.89129638671875, 1298.7928466796875, 0.9196962118148804], [519.576171875, 968.0533447265625, 0.8327004909515381], [497.0257568359375, 1133.4232177734375, 0.877360999584198], [481.99212646484375, 1276.242431640625, 0.9169248342514038], [480.1129150390625, 862.818115234375, 0.8726558685302734], [481.99212646484375, 757.5828857421875, 0.9131306409835815], [504.54254150390625, 674.89794921875, 0.8936622142791748], [500.78411865234375, 659.8643798828125, 0.9573098421096802], [549.6433715820312, 757.5828857421875, 0.9300024509429932], [572.1937866210938, 870.3349609375, 0.9395034313201904], [579.7105712890625, 975.5701904296875, 0.9278154373168945], [414.34088134765625, 757.5828857421875, 0.8962587714195251], [384.273681640625, 877.8516845703125, 0.9299138784408569], [384.273681640625, 983.0870361328125, 0.9225521683692932]], "keypoints_3d": [[0.00011357177572790533, -4.3044768972322345e-05, 0.8186586499214172], [0.0936516672372818, -0.06368296593427658, 0.8168273568153381], [0.04503236338496208, -0.004475868307054043, 0.40963801741600037], [0.11538084596395493, 0.15016400814056396, 0.0], [-0.09364932030439377, 0.06368304044008255, 0.8204939365386963], [-0.054762545973062515, 0.14730757474899292, 0.42772048711776733], [-0.025411054491996765, 0.3245426416397095, 0.03806495666503906], [-0.0006554634310305119, -0.05249069631099701, 1.0564631223678589], [-0.01368213165551424, -0.12149341404438019, 1.3038792610168457], [-0.06553129106760025, -0.14975431561470032, 1.4534919261932373], [-0.07273639738559723, -0.0451870858669281, 1.484086036682129], [-0.15405601263046265, -0.05994825437664986, 1.2695127725601196], [-0.22169850766658783, 0.005432157311588526, 1.0119078159332275], [-0.19742707908153534, -0.05209660157561302, 0.7784150838851929], [0.13977016508579254, -0.15153181552886963, 1.2430940866470337], [0.2367676943540573, -0.09030017256736755, 0.9781466722488403], [0.21135888993740082, -0.08324502408504486, 0.7569226622581482]], "title": "Prediction (0)", "bbox": [363.3894348144531, 594.4682006835938, 593.0779418945312, 1364.1890869140625, 0.9990841150283813]}

[LareinaM]

Hi, thanks for using MMPose. The keypoints_3d is the relative position on the image, you can check the pipeline in the config file for details.

[ghost]

Thanks for the advice. I checked the pipeline and realized that the class GetRootCenterdPose was an important clue as to the origin of the keypoints. However, I do not understand how it leads to the determination of coordinates in real space. Can you give me some clues about that?

[himanshukr028]

@ayasaki0407 how did you get the "track_id" for each instance..?