Support for `sim_arm64` (Apple Silicon) for iOS - Implementation steps included

[mathaeus]

System information (Please provide as much relevant information as possible)

• Programming language : Objective C / iOS
• Are you willing to contribute it (Yes/No): Yes

Describe the feature and the current behavior/state: Currently the iOS Mediapipe framework does not support compilation for arm64 Simulator (Apple Silicon). That means one can't compile an iOS app that includes the Mediapipe framework on a MacBook with M1 chip in the "native way" (unless you're targeting a physical device). Running Xcode in Rosetta mode is the only workaround at the moment.

Steps to add support for sim_arm64

ℹ️ I already implemented all the steps necessary in a fork and things work as far as I can tell.

• Below are all of the details.
• I can't open a PR right away since I think the openCV part needs to be solved in general first. Happy for feedback.

1. Add the config setting for the simulator to the Bazel config itself

2. Use a tensorflow release that supports sim_arm64

3. Use a pre-built openCV xcframework for iOS that supports sim_arm64

Since the official openCV repo does not yet come with pre-built xcframeworks that support sim_arm64 I built it myself from source with the command below and then uploaded it to a custom repo as a release asset. This step would need further exploration for openCV just also providing the pre-built xcframework.

// Checked out 'https://github.com/opencv/opencv/releases/tag/4.6.0' which is the newest release at the time of writing
python3 platforms/apple/build_xcframework.py --out build --iphoneos_archs arm64 --iphonesimulator_archs arm64,x86_64 --build_only_specified_archs

4. Update the Apple build rules in Bazel to the ones supporting xcframework (imports)

5. Adapt the openCV code for xcframework

---

Will this change the current api? How?

No

Who will benefit with this feature?

Anyone who is using the iOS Mediapipe framework (and is running on Apple Silicon - which will be sooner or later "everyone")

[mathaeus]

@sureshdagooglecom did you have some time to have a look at my suggested changes?

Is this something that the Mediapipe team can take care of or do you prefer me opening a PR? The open question about how to integrate OpenCV still needs to be answered. It seems like from the official OpenCV repo we can't expect support for pre-built xcframeworks anytime soon unfortunately.

[mathaeus]

@kuaashish pinging you in the hope to get some feedback :)

[sglim]

+1

[mathaeus]

Bumping this 😬

@sureshdagooglecom is this something that the Mediapipe team is planning to look into?

[shreyash-sharechat]

This issue is big inconvenience when you get a framework built out of Bazen and try to import it vanilla XCode project, when someone tries to run any of the targets that use MediaPipe as dependancy it will fail at linker phase with 'undefined symbols'. Only solution at this point is not support media pipe at all on simulators and wrap the Mediapipe API usage under environment flags (preprocessors). this.issue.bump()

[kuaashish]

Hello @mathaeus, We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediapPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.

You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions. Thank you

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.