mediapipe

mediapipe copied to clipboard

System information (Please provide as much relevant information as possible)

• Have I written custom code (as opposed to using a stock example script provided in Mediapipe):
• OS Platform and Distribution Linux Ubuntu 20.04
• [MediaPipe version] 0.8.6-0.8.10
• Bazel version:5.2.0
• Solution Pose
• Programming Language and version (Python):

Describe the expected behavior: A small error is in 0.8.4, pose_detection.tflite is 7.4M after 0.8.6 it become 3M. 0.8.7 use 7.4M will ok, small model will wrong. But a big issue is 0.8.9 0.8.10 version, maybe something wrong between the pose detection result and tracking gpu version.

I build gpu version python pose like in #2451, use the model complexity is 2 and apply to 1920x1080 video. I think the wrong result will appear on big videos.

I use the example code from official site, attentionly there is a time.sleep(0.1) before results = pose.process(image) , when use this code, the wrong result will more obvious.

A wrong result, normal speed. you can find landmark is lost . https://user-images.githubusercontent.com/29172715/181245598-474a4127-71d0-4af5-83f8-572d4f509fea.mp4 A wrong result, with sleep, you can find landmark is lost . you can find leg landmark has lost, I think the sleep affect the call of tracking. https://user-images.githubusercontent.com/29172715/181245724-7bcce626-121f-4ced-ac23-e757c9f652af.mp4 A right result use cpu. https://user-images.githubusercontent.com/29172715/181245965-cf17dd68-7d89-4415-a04f-cbfefc8a4488.mp4

The error happened on 1920x1080 video, small video may be fine.

test video

https://user-images.githubusercontent.com/29172715/181246496-0d1eff54-9a90-48fe-befd-a93eec9ab163.mp4

import time

import cv2
import mediapipe as mp
mp_drawing = mp.solutions.drawing_utils
#mp_drawing_styles = mp.solutions.drawing_styles
mp_pose = mp.solutions.pose


cap = cv2.VideoCapture('kunkun_cut.mp4')
with mp_pose.Pose(
    static_image_mode=False,
    model_complexity=2,
    smooth_landmarks=False,
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5) as pose:
  while cap.isOpened():
    success, image = cap.read()
    if not success:
      print("Ignoring empty camera frame.")
      # If loading a video, use 'break' instead of 'continue'.
      continue

    #time.sleep(0.1)
    # To improve performance, optionally mark the image as not writeable to
    # pass by reference.
    image.flags.writeable = False
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    start_time = time.time()
    results = pose.process(image)
    print('mediapipe fps', 1 / (time.time() - start_time))

    #Draw the pose annotation on the image.
    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    mp_drawing.draw_landmarks(
        image,
        results.pose_landmarks,
        mp_pose.POSE_CONNECTIONS,
    )
        #landmark_drawing_spec=mp_drawing_styles.get_default_pose_landmarks_style())
    # Flip the image horizontally for a selfie-view display.
    cv2.imshow('MediaPipe Pose', cv2.flip(image, 1))
    if cv2.waitKey(5) & 0xFF == 27:
      break
cap.release()

wrong result example 2

I saw you have this setting in your code: min_detection_confidence=0.5

So I guess for some of the landmarks, there visibility score is lower than 0.5, that's why they are not displayed.

There is a visibility field in each landmark output, you can print them out and check their values.

No, not for this reason. I use 0.1 also has this problem. When I use cpu a pip install version, results is correct.

Hello @linyu0219, We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediapPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions. Thank you

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

This issue was closed due to lack of activity after being marked stale for past 7 days.

Are you satisfied with the resolution of your issue? Yes No