mediapipe
mediapipe copied to clipboard
ImageSegmentation new option to provide only the contours points to improve performance
Have I written custom code (as opposed to using a stock example script provided in MediaPipe)
No
OS Platform and Distribution
Android 14
MediaPipe Tasks SDK version
No response
Task name (e.g. Image classification, Gesture recognition etc.)
Image Segmentation
Programming Language and version (e.g. C++, Python, Java)
Web Javascript -> tasks_vision 0.10.17
Describe the actual behavior
The functionality is correct but the performance obtaining the pixels is really slow
Describe the expected behaviour
I'd like to have an option to only get the contours points
Standalone code/steps you may have used to try to get what you need
In my use case I'm using multiclass selfie segmenter, and what I do when I receive the Uin8Array with the confidence values for each MPMask is findContours with opencv. In this case I don't really need all the points from mediapipe, and from what I saw in with the performance profiler, the segmentation synchronous call takes 30ms, and the readPixels when executing getAsFloat32Array or getAsUint8Array takes 80-100ms.
With this context, to improve the performance in multiple cases I'm proposing/requesting for a new option that instead of getting all the points for a category for example, get only the points that define the contours. (I'm saying this because from what I know mediapipe uses opencv underneath and I imagine is easier than developing a new way of getting the contours)
I think returning much less points from GPU to CPU would improve the performance a lot, even that the computation can increase because now after the segmentation you'll be also finding the contours.
I'm not sure if the proposals should be git issues or is this any other better way. Sry in that case
Other info / Complete Logs
No response