MediaPipeUnityPlugin
MediaPipeUnityPlugin copied to clipboard
homuler
When changing settings the camera hangs in Async mode on Android (Samsung S22 Ultra and other Galaxy models)
Description
I am running pose tracking from MediaPipeUnityPlugin-all.zip on Android in Async mode for pose tracking. When I am starting demo first time, it immediately show camera feed but start showing landmarks only after 10-15 seconds when mediapipe initializes (or something). When I am in Sync mode app freezes during that time. But I use Async, I wait and that is not a problem. When I change camera or something in settings, the app freezes until mediapipe is initializes again, but I'd like to see at least camera feed working like during startup and only wait for mediapipe to initialize.
I was debugging and noted that it hangs here. It doesn't return from that function. https://github.com/homuler/MediaPipeUnityPlugin/blob/6b8c6743f23539f7604e74dc260b01e0f58f1707/Assets/MediaPipeUnity/Samples/Common/Scripts/ImageSource/TextureFrame.cs#L278
I compared it to PC and noted that on PC _glSyncToken is null but on Android it isn't. So I finally commented out below line and I now I can change settings on Android without any extra blocking on the ui/camera. Still need wait for Mediapipe to initialize but that is fine. https://github.com/homuler/MediaPipeUnityPlugin/blob/6b8c6743f23539f7604e74dc260b01e0f58f1707/Assets/MediaPipeUnity/Samples/Common/Scripts/ImageSource/TextureFramePool.cs#L121
I can't see where I can go any further, also don't like idea of commenting that out so perhaps you can find solution for this. Regards
Thank you for reporting the issue.
So I finally uncommented out below line and I now I can change settings on Android without any extra blocking on the ui/camera.
I'm not sure, but it looks like a bug to me.
I'm not sure, but it looks like a bug to me.
Sorry. I meant that I commented out the line cause there was no comment before. Not removed the comment from already commented code.
I meant that I commented out the line cause there was no comment before.
Never mind. I know that.
If I knew that, I would have fixed it already. In the first place, I think I don't understand exactly what the problem is.
related to https://github.com/homuler/MediaPipeUnityPlugin/issues/768#issuecomment-1285330371
Maybe GlSyncPoint#WaitOnGpu should be called instead.
cf. https://github.com/google/mediapipe/blob/68ba9a6ebf27e9c6c9e4500f7e1b68dd2db8c05b/mediapipe/java/com/google/mediapipe/components/ExternalTextureConverter.java#L603