mediapipe

mediapipe copied to clipboard

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

Linux

MediaPipe version

No response

Bazel version

No response

Solution

Pose

Programming Language and version

Python

Describe the actual behavior

When I use solutions.drawing_utils.draw_landmarks(), some joints are missing

Frame1: 
23th joint: NormalizedLandmark(x=0.5970980525016785, y=0.9964555501937866, z=-0.003643944626674056, visibility=0.8678634762763977, presence=0.5461540818214417) 
24th joint: NormalizedLandmark(x=0.4613693952560425, y=0.9909225702285767, z=0.0033864073920994997, visibility=0.8655446767807007, presence=0.5349380373954773)

Frame2: 
23th joint: NormalizedLandmark(x=0.5915066599845886, y=1.009007215499878, z=0.010031683370471, visibility=0.9218407273292542, presence=0.6105213761329651) 
24th joint: NormalizedLandmark(x=0.45220595598220825, y=0.9995362758636475, z=-0.010209781117737293, visibility=0.9436008930206299, presence=0.6531380414962769)

These are 23th(L_hip) and 24th joints' properties of frame 1 and frame2, where the the L_HIP's visualization presents in the first frame but it is not visualized in the second frame. I don't know the reason for this because the visibility and presence are even higher in frame2.

Describe the expected behaviour

What is the visibility/confidence threshold for visulising the joints?

Standalone code/steps you may have used to try to get what you need

def draw_landmarks_on_image(rgb_image, detection_result):
    pose_landmarks_list = detection_result.pose_landmarks
    annotated_image = np.copy(rgb_image)

    # Loop through the detected poses to visualize.
    assert len(pose_landmarks_list) == 1
    for idx in range(len(pose_landmarks_list)):
        pose_landmarks = pose_landmarks_list[idx]

        # Draw the pose landmarks.
        pose_landmarks_proto = landmark_pb2.NormalizedLandmarkList()
        pose_landmarks_proto.landmark.extend([
        landmark_pb2.NormalizedLandmark(x=landmark.x, y=landmark.y, z=landmark.z) for landmark in pose_landmarks
        ])
        solutions.drawing_utils.draw_landmarks(
        annotated_image,
        pose_landmarks_proto,
        solutions.pose.POSE_CONNECTIONS,
        solutions.drawing_styles.get_default_pose_landmarks_style())

    lmList = []
    print(pose_landmarks_list[0][23], pose_landmarks_list[0][24])
    for idx, landmark in enumerate(pose_landmarks_list[0]):
        h, w, c = rgb_image.shape
        px, py = int(landmark.x * w), int(landmark.y * h)
        lmList.append([px, py, 1])

    mediapipe_body_joints = np.array(lmList)

    return annotated_image, mediapipe_body_joints


body_base_options = python.BaseOptions(model_asset_path='pose_landmarker_heavy.task')
options = vision.PoseLandmarkerOptions(
        base_options=body_base_options,
        # output_segmentation_masks=True)
        running_mode=VisionRunningMode.IMAGE)
detector = vision.PoseLandmarker.create_from_options(options)

detection_result = detector.detect(mp.Image.create_from_file(prefix.replace('videos-new', 'images') + str(i+1).zfill(6) + '.png'))
                vis, mediapipe_body_joints = draw_landmarks_on_image(vis, detection_result)

Other info / Complete Logs

No response