GPU Buffer usage with Tasks system

[arnoldy]

Plugin Version or Commit ID

v0.12.0

Unity Version

2021.3.8

Your Host OS

WIndows 10.

Target Platform

Android, iOS

Description

Hi, @homuler, thank you for your effort developing this plugin.

Is there a way to use GPUBuffer for Android/iOS (GPU) with tasks solutions to avoid copying texture back to CPU? As some mid-grade devices like Motorolla Edge having hard times preprocessing textures, mid-grade iOS devices having a delay in async mode because of async readback is not always finishing on same frame.

I understand its wrong but what i have tried it using gpubuffer ptr to make Image instance to be used in DetectAsync method from yours tasks implementation.

Thank you in advance for any guidance and tips.

Code to Reproduce the issue

Wrong solution i tried )

faceTextureFrame.ReadTextureOnGPU(faceTex);
var glContext = GlContext.GetCurrent(); 
var glTextureBuffer = new GlTextureBuffer(faceTextureFrame.GetTextureName(), faceTextureFrame.width, faceTextureFrame.height,
faceTextureFrame.gpuBufferformat, faceTextureFrame.onDeletion, glContext); 
var gpuBuffer = new GpuBuffer(glTextureBuffer); 
Image im = new Image(gpuBuffer.mpPtr);
_faceRunner.ProcessFrameAsyc(im);

where ProcessFrameAsyc is just a wrapper for DetectAsync

crash log GLBufferCrashLog.txt

Additional Context

No response

[homuler]

Is there a way to use GPUBuffer for Android/iOS (GPU) with tasks solutions to avoid copying texture back to CPU?

When I started to port the Task API, there was no way to do that as far as I know. However, for now, there's an API to specify the GpuResources, so I'll consider supporting that option. cf. https://github.com/google/mediapipe/blob/4f4f107a8b26091e7e790ff3e3ea1fa4656441b0/mediapipe/tasks/python/core/pybind/task_runner.cc#L83