ADD mediapipe pose real-world 3d coordinates

[kyakuno]

blazepose-3dは下記のissueでtf2onnx変換している。 https://github.com/axinc-ai/ailia-models/issues/496 同様に、POSE_WORLD_LANDMARKSを変換したい。 https://google.github.io/mediapipe/solutions/pose.html#pose_world_landmarks https://google.github.io/mediapipe/solutions/pose.html 前処理と後処理もPythonで実装する必要がある。

[kyakuno]

Getting the Pose World Landmarks in the Pose Tracking Android Demo https://github.com/google/mediapipe/issues/3013

[kyakuno]

下記にモデルがないので、後処理だけの実装な可能性がある。 https://google.github.io/mediapipe/solutions/models.html#pose

[ooe1123]

[推論処理流れ]

mediapipe/modules/pose_landmark/pose_landmark_cpu.binarypb
PoseLandmarkCpu

    GateCalculator
      - image_for_pose_detection
      ↓
    PoseDetectionCpu
      - pose_detections
      ↓
    SplitDetectionVectorCalculator
      - pose_detection
      ↓
    PoseDetectionToRoi
      - pose_rect_from_detection
      ↓
    MergeCalculator
      - pose_rect
      ↓
    mediapipe/modules/pose_landmark/pose_landmark_by_roi_cpu.pbtxt
    PoseLandmarkByRoiCpu
      ImageToTensorCalculator
        - input_tensors
        ↓
      InferenceCalculator
        - output_tensors
        ↓
      mediapipe/modules/pose_landmark/tensors_to_pose_landmarks_and_segmentation.pbtxt
      TensorsToPoseLandmarksAndSegmentation
          - tensors
          ↓
        SplitTensorVectorCalculator
          - landmark_tensor
          - world_landmark_tensor
          ↓
        GateCalculator
          - ensured_world_landmark_tensor
          ↓
        TensorsToLandmarksCalculator
          - all_world_landmarks
          ↓
        SplitLandmarkListCalculator
          - world_landmarks_without_visibility
          ↓
        VisibilityCopyCalculator
          - world_landmarks
          ↓
        - roi_world_landmarks
        ↓
      mediapipe/modules/pose_landmark/pose_landmarks_and_segmentation_inverse_projection.pbtxt
      PoseLandmarksAndSegmentationInverseProjection
          - roi_world_landmarks
          - roi
          ↓
        WorldLandmarkProjectionCalculator
          - world_landmarks
          ↓
        - landmarks
        - world_landmarks
      ↓
    PoseLandmarkFiltering
      ↓
      - pose_world_landmarks

[kyakuno]

tfliteのエクスポートメモ

tflite2tensorflow \
  --model_path pose_landmark_heavy.tflite \
  --flatc_path ~/tflite2tensorflow/flatc \
  --schema_path ~/tflite2tensorflow/schema.fbs \
  --output_pb
tflite2tensorflow \
  --model_path pose_landmark_heavy.tflite \
  --flatc_path ~/tflite2tensorflow/flatc \
  --schema_path ~/tflite2tensorflow/schema.fbs \
  --output_onnx --onnx_opset 12