mediapipe icon indicating copy to clipboard operation
mediapipe copied to clipboard

Android/Java PoseLandmarker unnecessary []byte garbage

Open calbot opened this issue 1 year ago • 2 comments

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

Yes

OS Platform and Distribution

Android

MediaPipe Tasks SDK version

No response

Task name (e.g. Image classification, Gesture recognition etc.)

PoseLandmarker

Programming Language and version (e.g. C++, Python, Java)

Kotlin/Java

Describe the actual behavior

AndroidPacketGetter.copyRgbToBitmap calling ByteBuffer.allocateDirect creating massive amounts of byte array GC

Describe the expected behaviour

Reuse byte[] on the same thread in AndroidPacketGetter.copyRgbToBitmap

Standalone code/steps you may have used to try to get what you need

AndroidPacketGetter is misusing ByteBuffer.allocateDirect which is meant for long lived ByteBuffers. It seems to be a major memory allocation/GC area for PoseLandmarker.

Maybe these should be threadlocal cached by size (width * height) or some other way to avoid so many allocations. Or maybe the byte buffer should be passed in from PoseLandmarker.createFromOptions where AndroidPacketGetter.getBitmapFromRgb is called.

Other info / Complete Logs

No response

calbot avatar Sep 14 '24 04:09 calbot

I did a test on dagster+. Turns out dagster+ has a hardcoded dependency on setup.py or requirement.txt.

Screenshot 2024-10-19 at 07 52 43 Screenshot 2024-10-19 at 08 09 46

The responsible code is here. Not sure if it is justifiably pessimistic about file dependency. https://github.com/dagster-io/dagster-cloud/blob/main/dagster-cloud-cli/dagster_cloud_cli/commands/serverless/init.py#L534-L545

dbrtly avatar Oct 18 '24 21:10 dbrtly

@PedramNavid @cmpadden How do you feel about installing on dagster+ with uv to replace pex? https://docs.astral.sh/uv/concepts/tools/

https://github.com/dagster-io/dagster-cloud-action/pull/202

After above PR, I will do PR on dagster-cloud to remove: https://github.com/dagster-io/dagster-cloud/blob/main/dagster-cloud-cli/dagster_cloud_cli/commands/serverless/init.py#L534-L545 and extend https://github.com/dagster-io/dagster-cloud/blob/main/dagster-cloud-cli/dagster_cloud_cli/commands/serverless/Dockerfile#L6

I'd like to split the build code from the deploy code. It feels like the github-action it could be simpler with stricter separation of build/push-to-registry/deploy.

dbrtly avatar Oct 19 '24 01:10 dbrtly