mediapipe

mediapipe copied to clipboard

Please make sure that this is a bug and also refer to the troubleshooting, FAQ documentation before raising any issues.

System information (Please provide as much relevant information as possible)

• Have I written custom code: yes
• OS Platform and Distribution: Android 10
• Mobile device: Huawei P30 Pro
• Programming Language and version: C++
• [MediaPipe version]: v0.8.8 commit: cf101e6 from 2021-11-13 05:10
• Bazel version: 3.7.2
• Solution: Pose
• NDK version: 21.3.6528147
• SDK version: 30.0.3

current behavior: Our app using mediapipe runs fine on many Android devices, but on a particular device (Huawei P30 Pro), we get the following error:

2022-01-24 13:39:26.259 24512-24728/org.aisc.ailib.example.app_example_2 E/FrameProcessor: Mediapipe error: 
    com.google.mediapipe.framework.MediaPipeException: internal: Graph has errors: 
    Calculator::Process() for node "poselandmarkgpu__poselandmarkbyroigpu__poselandmarksandsegmentationinverseprojection__InverseMatrixCalculator" failed: ; Inverse matrix cannot be calculated.le_mediapipe/mediapipe/calculators/util/inverse_matrix_calculator.cc:38) 
        at com.google.mediapipe.framework.Graph.nativeMovePacketToInputStream(Native Method)
        at com.google.mediapipe.framework.Graph.addConsumablePacketToInputStream(Graph.java:395)

The app crashes after throwing the above mentioned error.

We found two fixes for mediapipe to prevent the app from crashing. You can check them in my forked mediapipe repository here

The fix, prevents the app from crashing the detected pose now becomes weird and sometimes gets lost for a couple of seconds. Sometimes the detection "catches itself" again after a while but sometimes it's also gone forever. However the detection accuracy on the above mentioned smartphones still differs significantly from that on other devices. It is unclear why this bug only appears on some devices. The bug appears also if instead of the live camera feed we input frames from a prerecorded video.

expected behavior: Mediapipe's pose detection accuracy should work similar on different devices with video frames as the input.

Standalone code to reproduce the issue: not possible. we test it inside our own app, but the error clearly comes from mediapipe as it should not be possible that the app crashes entirely no matter what the input images to the mediapipe graph look like

Other info / Complete Logs : ... not sure what to put here, can update this section on demand...

Thanks for sharing us complete details of the above issue with with fixes. Our engineer team will review it and will let you know the status on the same.

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you.

This issue should not be stale in my opinion...
Meanwhile I tried recompiling with some different versions of Mediapipe

• tag 0.8.8 , commit: 33d683c67100ef3db37d9752fcf65d30bea440c4
• tag 0.8.9, commit: e6c19885c6d3c6f410c730952aeed2852790d306 and testing with the default pose tracking demo apk: "posetrackinggpu.apk"

To reiterate:

• All of the following observations only happened on the Huawei P30 Pro!
• All other devices that I tested work fine

Observations:

• 0.8.8 does not work at all
• 0.8.9 works better, but there are still some issues:
  • the app does not crash anymore, however the accuracy of the pose-estimation is significantly worse on than on other Android devices
  • the skeleton gets lost quite frequently and takes forever to catch itself again
  • sometimes the skeleton fits to my face "upside-down"
  • other times the skeleton wanders around fitting to nothing / background

Since this annoying behaviour only occurs on my Huawei P30 Pro, I guess the problem might still be related to #2602

Just had a similar error on windows.

windows, python 3.7, mediapipe pose model complexity 1

WARNING: Logging before InitGoogleLogging() is written to STDERR E20220625 02:25:20.911455 32260 calculator_graph.cc:804] INTERNAL: CalculatorGraph::Run() failed in Run: Calculator::Process() for node "poselandmarkbyroicpu__poselandmarksandsegmentationinverseprojection__InverseMatrixCalculator" failed: ; Inverse matrix cannot be calculated.tors/util/inverse_matrix_calculator.cc:38)

I run into the same error as @alexiscatnip with Windows, Python 3.7, model complexity 2

Hello @mgarbade, We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediapPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions.

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

This issue was closed due to lack of activity after being marked stale for past 7 days.

Are you satisfied with the resolution of your issue? Yes No