mediapipe

mediapipe copied to clipboard

Please make sure that this is a bug and also refer to the troubleshooting, FAQ documentation before raising any issues.

System information (Please provide as much relevant information as possible)

• Have I written custom code (as opposed to using a stock example script provided in MediaPipe): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04, Android 11, iOS 14.4): Windows 11 Version 21H2 (Build 22000.466)
• Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
• Browser and version (e.g. Google Chrome, Safari) if the issue happens on browser: 97.0.4692.99 (Official Build) (64-bit) (cohort: Stable)
• Programming Language and version ( e.g. C++, Python, Java): JS
• MediaPipe version: [email protected]
• Bazel version (if compiling from source):
• Solution ( e.g. FaceMesh, Pose, Holistic ): Hands
• Android Studio, NDK, SDK versions (if issue is related to building in Android environment):
• Xcode & Tulsi version (if issue is related to building for iOS):

Describe the current behavior: The left/right handedness classification of MP Hands isn't stable or reliable, notably when the hand is partially out of frame or partially intersected with the other hand.

Describe the expected behavior: Expected that the handedness classification is reliable and stable, it shouldn't change from one frame to the next for the same physical hand.

Standalone code to reproduce the issue: Provide a reproducible test case that is the bare minimum necessary to replicate the problem. If possible, please share a link to Colab/repo link /any notebook: Just log the results.multiHandedness[handIndex] and try MP Hands, put your hand near the edge of the display or near your other hand when both hands are shown.

Here is a scaffold (not the full repro solution).

function onResults(results) { canvasCtx.save(); canvasCtx.clearRect(0, 0, canvasElement.width, canvasElement.height);

if (
  results.multiHandLandmarks &&
  results.multiHandedness &&
  results.multiHandWorldLandmarks &&
  results.multiHandLandmarks[0] &&
  results.multiHandWorldLandmarks[0]
) {
  // Outer: Iterate over detected hands (i.e. 0, 1 or 2)
  for (let handIndex = 0; handIndex < results.multiHandLandmarks.length; handIndex++) {

    const classification = results.multiHandedness[handIndex];
    console.log(classification);
  }
}

}

Other info / Complete Logs : Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached

E.g. Physical hand barely moved, but here is the console output of the handedness label from a sequence of detections.

{ "index": 1, "score": 0.502685546875, "label": "Right" } { "index": 1, "score": 0.62939453125, "label": "Right" } { "index": 1, "score": 0.5009765625, "label": "Right" } { "index": 0, "score": 0.638671875, "label": "Left" } { "index": 0, "score": 0.74462890625, "label": "Left" }

Hi @jsegeren, Can you illustrate the above with any reference video on the above query. Thanks!

Sure, here's a cropped video so that I'm not including IP. https://user-images.githubusercontent.com/2702674/152470451-02dd7a61-9cb2-4d9a-a5f1-0383a1ae882c.mp4

any update on this? I had similar issue when running mediapipe hands in Touchdesigner (python)

Any update?
In my practices, except partially out of frame and partially hands intersected, the handedness would be confused as long as hand is close to nothing detection. In this situation, for example, there is a left(or right) hand exactly, MP would return two left hands or two right hands or a left hand and a right hand, the issue #3902 is one of this. All above is about hand confusion as there is a left(or right) hand exactly. Another bug is that there would be three hands detected while i set MAX_NUM_HANDS = 2 as there is only one in front of the camera, reference #2319

Just now, MP mistook my face for a hand. The code I used is at https://google.github.io/mediapipe/solutions/hands.html#python-solution-api, and i didn't modify anything except delete section of static image.

Hello @jsegeren, We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediaPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions.

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

This issue was closed due to lack of activity after being marked stale for past 7 days.

Are you satisfied with the resolution of your issue? Yes No