Mediapipe tasks-vision iOS 17+ inside Web Workers not working, despite iOS 17+ already support it.

[SystemPanic]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

None

OS Platform and Distribution

iOS 17.4.1 / iPadOS 17.4.1

MediaPipe Tasks SDK version

0.10.12 (latest on npm)

Task name (e.g. Image classification, Gesture recognition etc.)

All

Programming Language and version (e.g. C++, Python, Java)

Javascript

Describe the actual behavior

OffscreenCanvas is not used inside Web Workers when iOS 17+ supports it. It tries to create a normal canvas via document.createElement("canvas"), which obviously does not works inside a Worker.

Describe the expected behaviour

tasks-vision creates the OffscreenCanvas instead of DOM canvas inside Web Workers

Standalone code/steps you may have used to try to get what you need

Load tasks-vision (for example, for Face Landmarks mesh) inside a Web Worker on iOS 17+. It will throw:

 Can't find variable: document

I can make tasks-vision to work perfectly if I change

 typeof OffscreenCanvas !== 'undefined' && !isWebKit()

to 

 typeof OffscreenCanvas !== 'undefined'

on graph_runner.ts:111 and vision_task_runner.ts:47

Also, but with this I'm not completely sure, iOS 17+ has full gl.FLOAT support, so checking if isIOS on mask.ts:306 is not necessary anymore. I've made a few test with Mesh / Hand models forcing isIOS to return false and it works correctly, but as I'd said, maybe it requires further testing.

Other info / Complete Logs

No response

[kuaashish]

Hi @SystemPanic,

Could you kindly provide the detailed steps you are following from our documentation, or alternatively, share the standalone code with us? This will help us to reproduce the issue and gain a better understanding of it.

Thank you!!

[schmidt-sebastian]

We will fix this and verify on iOS 17.

[kuaashish]

HI @SystemPanic,

We've launched the latest version 0.10.13, which includes support for OffscreenCanvas in iOS 17+, as detailed in the release notes ("Update WebGL2 on OffscreenCanvas support check to include Safari 17+"). Could you please verify now? If the issue persists on your end, please inform us.

Thank you!!

[arcinston]

@SystemPanic could you help guide how you were able to import tasks-vision on a web worker

i am not able to find any resource on how to go about doing it and tried a lot but no success

[kuaashish]

Hi @arcinston,

Currently, we do not have any examples demonstrating web workers with our task-vision. I hope @SystemPanic can assist you with this. If I find any resources showing how to run web workers with task-vision, will inform you.

In the meantime, @SystemPanic, could you please confirm if you can now run Mediapipe inside the web worker on iOS 17+? We have resolved this issue.

Thank you!!

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No