MLKit iOS on GPU/NPU or reduce CPU load?

[igorzharii]

Hi,

For the last couple of days I'm trying to figure out running MLKit for Pose detection on an iPhone's GPU or NPU for higher efficiency than the CPU. From what I understand there is no GPU/NPU processing possible for now, is there any ETA for that? Or maybe there is a way of reducing the CPU load without reducing quality? (Using the .fast model in practice doesn't change the CPU load much, and there are no other visible things to adjust) The reason behind my question is that on longer recognizing streaks (30-60+ minutes) the phone gets noticeably hot which could probably be avoided with processing done on the GPU/NPU.

Thanks

[ksulaimon]

Hey igorzharii,

We are currently working on an optimized version of our pose detector that uses GPU processing and are hoping to have a solution for this early next year. We are aware that running longer sessions causes the CPU to heat up etc, and we are hopeful that our move to GPU will help with this.

In the meantime, have you tried only running the pose detector when absolutely necessary? I'm not sure what your use case is, but turning off the pose detector intermittently when you don't directly need the pose input definitely helps with the thermal/performance issues.

[igorzharii]

Yep, actually pausing and resuming processing for now. Thanks for the update, looking forward to it!

[DmytroUsenko]

Hey igorzharii,

We are currently working on an optimized version of our pose detector that uses GPU processing and are hoping to have a solution for this early next year. We are aware that running longer sessions causes the CPU to heat up etc, and we are hopeful that our move to GPU will help with this.

In the meantime, have you tried only running the pose detector when absolutely necessary? I'm not sure what your use case is, but turning off the pose detector intermittently when you don't directly need the pose input definitely helps with the thermal/performance issues.

Hey @ksulaimon . Do you have any updates about this issue?

[Throvn]

Hey, I don't want to pressure, but I couldn't find it in the releasenotes. Is it just not a high priority right now, or can one somehow help?

[bcdj]

Unfortunately due to other more important work, this feature hasn't been prioritized yet.

[TelOkan]

Hi everyone. I suggest to use mediaPipe. It is using gpu to poseDetection. I think so it will help at performance.

[linnaok]

Hi everyone. how to use MLKit on NPU?