Inference time decreasing after a certain delay using GPUBuffer packets running in GL Context

[ABazL]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Android 13

MediaPipe version

0.8.10

Bazel version

5.2.0

Solution

Pose

Programming Language and version

No response

Describe the actual behavior

Hello,

I'm using MediaPipe v0.8.10 on a Unity prototype project through MediaPipeUnityPlugin C# wrapper (v0.10.3) and I'm experiencing a weird behaviour that doesn't seem linked to the C# wrapper itself.

Issue is that by using GPUBuffer packet instead of ImageFrame packet, I see an inference processing time improvement only after a certain delay that vary.

For example on a Redmi Note 10 Pro, during X seconds after app started I'll have an avg inference time about 45/50ms and after this delay of X seconds, avg inference time is about 15ms.

I did some profiling with Snapdragon Profiler and it seems that GPU clock speed isn't at its maximum speed from the beggining of the app (nor beginning of MediaPipe starting to process):

I wanted to know if it's a known behaviour and if it's the case, is there a solution to force GPU clock speed or at least to say at a specific point at runtime that we need as many GPU resources possible (when starting MediaPipe graph) ?

You can check full discussion I started from the Github of the MediaPipe C# wrapper from Homuler here for more context: https://github.com/homuler/MediaPipeUnityPlugin/issues/1140

Thanks!

Describe the expected behaviour

Expected behaviour is a pretty constant inference time using GPUBuffer sent from a GL Context.

Standalone code/steps you may have used to try to get what you need

No response

Other info / Complete Logs

No response

[kuaashish]

Hi @gkarpiak,

Could you please have look into this issue? Thank you!!

[ABazL]

Hi, is there any update on this @kuaashish, @gkarpiak ?

Thanks!

[ABazL]

Hi again, is there any update on this @kuaashish, @gkarpiak ? Thanks

[gkarpiak]

MediaPipe framework doesn't provide a way to change GPU frequency on the device. Device itself can change the frequency depending on various factors/rules. E.g. in your case, possibly you run heavy work for some time and device decides to increase the frequency on demand; also, it can be so that after running for a longer time, device would heat up and decrease the frequency not to overheat.

So, you may need to research possible ways to somehow lock the performance. From a quick look you may want to read through https://developer.android.com/games/optimize/overview

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No