Attempting to rebuild the CocoaPod equivalent of iOS MediaPipeTaskVision.xcframework & MediaPipeTasksCommon.xcframework

[pixeled]

OS Platform and Distribution

MacOS - 13.6.4

Compiler version

No response

Programming Language and version

OjbC

Installed using virtualenv? pip? Conda?(if python)

git clone

MediaPipe version

0.10.9

Bazel version

6.1.1

XCode and Tulsi versions (if iOS)

Xcode 15.2

Android SDK and NDK versions (if android)

No response

Android AAR (if android)

None

OpenCV version (if running on desktop)

whatever MediaPipe wants

Describe the problem

There is not a single README, or document that says what bazel command to build the framework

Complete Logs

I don't want to run hello_world.

I just want to build the same iOS xcode framework that is in the CocoaPod MediaPipeTaskVision.xcframework (and probably MediaPipeTasksCommon.xcframework), but with a -DBUILD_opencv_world=ON .. that's it. 

nothing magical.

as far as I have been looking, and trying, literally for the last 3 hours, there are no simple instructions on how to do this.

there's "use tulsi" and then the page that says "tulsi is unsupported, use rules_xcodeproj" - but no one has what you need broken down - everyone, I think is assuming that everyone is already a Bazel expert. Well - I've been coding iOS for 16 years, and MacOS for 40 years, and TODAY is the first I've ever heard of Bazel.

So.. Google.. big company. Add a section to your README that says "if you want to make MediaPipeTaskVision.xcframework & MediaPipeTasksCommon.xcframework, these are the steps"

[kuaashish]

Hi @pixeled,

We use a build script to create our CocoaPods targets, but we have not tested this specific scenario yet. Please use the freely available MediapipeTasksVision pod.

Thank you!!

[pixeled]

if these could be added to the repo, or made public somewhere that would unblock me quickly. (and as told by the internet searches, hundreds of other people too)

[pixeled]

Hi @pixeled,

We use a build script to create our CocoaPods targets, but we have not tested this specific scenario yet. Please use the freely available MediapipeTasksVision pod.

Thank you!!

As stated in the original post, I can not use the freely available MediaPipeTaskVision pod, because it contains some but not all OpenCV calls -

It includes just enough OpenCV to prevent me from including OpenCV framework, but not enough to allow me to use the OpenCV from within MediaPipeTaskVision

I would like to make an UBER MediaPipeTaskVision that includes the “world” OpenCV

So I’m asking again.

Could you please include / send / post the build script that will build the CocoaPods target, because the instructions on how to use bazel / bazelisk / tulsi / etc are all either outdated, incomplete, or just broken (python3 v python etc)

[kuaashish]

Hi @pixeled,

Is this 5051(Comment) useful for the situation you are addressing? If not, please inform us, and we will assign this matter to the appropriate contact.

Thank you!!

[pixeled]

I’ve actually run across those, today ironically - Upon trying those files, including the .a file - It compiles, and links, but then when you try to use the Face Landmarks it proceeds to throw exceptions, requiring first two NSString methods from non-existing categories, then throws an exception for a missing MPP object.

I have the MediePipe code from github. I have located where the config open cv is limited the inclusion of the OpenCV modules necessary.

All I want is the build scripts that make the CocoaPod version of MediaPipeTaskVision & MediaPipeTasksCommon. Because, as previously discussed, there’s not an existing system that can build the MediaPipe github repository even with all the build tools installed. (Bazel version 2.0 is documented as required, even though Bazel 6.x is current, and 2.x is not even available)

I would be just as happy with ios instructions that would just work - https://developers.google.com/mediapipe/framework/getting_started/ios just doesn’t - i can get through to the “Hello World” but that doesn’t get me the CocoaPod equivalent frameworks

just something. ANYTHING that can create the .frameworks would be great - and steps that can be followed on at least MacOS 13.6.4 (or later) and Xcode 15.2

[kuaashish]

Hi @priankakariatyml,

Could you please have look into this issue too?

Thank you!!

[priankakariat]

We build the OpenCV iOS xcframework from source while building the iOS MediaPipe frameworks using Bazel. Please check the Bazel installation instructions here.

This is incorporated into the pipeline that builds the MediaPipe frameworks. You have to clone the repo and modify the the genrule in the Bazel BUILD file:thirdaprty/opencv_ios_source.BUILD ]. Specifically the arg cmd in the genrule .

Internally this script runs the following OpenCV Python script that builds iOS xcframeworks: https://github.com/opencv/opencv/blob/4.5.3/platforms/apple/build_xcframework.py

We exclude many modules but don't include anything other than what the OpenCV script includes by default. Not sure if there is a command line arg to include a module. You can check the source of the OpenCV script to see what can be done.

After modifying the genrule run mediapipe/tasks/ios/build_ios_framework.sh command: FRAMEWORK_NAME=MediaPipeTasksCommon MPP_BUILD_VERSION=0.1.1 'mediapipe/tasks/ios/build_ios_framework.sh'

Repeat the previous command with changes for MediaPipeTasksVision framework.

You can find the frameworks in the archives built by the bash script. By default they will be created in your home folder.

Once you have both the frameworks, you have to include them in your Xcode project.

For MediaPipeTasksCommon, you have to unzip MediaPipeTasksCommon/0.1.1/ MediaPipeTasksCommon-0.1.1.tar.gz and copy the frameworks folder to you Xcode Project. This folder contains two sub folders, graph_libraries and the MediaPipeTasksCommon.xcframework. MediaPipeTasksCommon.xcframework must be added to your target like any xcframework. But ensure that you don't link the graph_libraries to the project. It should be like a file whose reference is deleted from the Xcode Project's target but the file will still be present in the folder structure of your project. We use the Other Linker Flags to force load these graph libraries using their path.

Similarly, add MediaPipeTasksVision.xcframework to your project too.

Our CocoaPods installation modifies the Other Linker Flags in the project's build settings. You can check all the flags that get added to your target's build settings after the MediaPipeTasksVision CocoaPod gets installed. You have to mimic this exactly to avoid the errors. There are flags which use the file paths to the graph libraries. Ensure that these are set to the exact location where your graph libraries are present. For simulator and device, we load different graph libraries. So please pay attention to the Any iOS Simulator SDK and Any iOS SDK under the Other Linker Flags. A lot of our functionality depends on all the flags being set correctly.

P.S: The error you are seeing is because the -ObjC flag is not present.

Also if your graph libraries are not linked correctly, your task initialisation will throw an error.

There are many errors you may face in the Bazel BUILD process especially since you are trying to customise the BUILD. If you face any specific issue when trying this out, you can share your errors. We can help you along the process.

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No

[WaterBoiledPizza]

I tried to follow the steps to build the pod but ran into the following error:

INFO: Analyzed target //mediapipe/tasks/ios:MediaPipeTasksCommon_framework (219 packages loaded, 15981 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_user/cabae5a58991828d2e35db74d5b63d10/external/ios_opencv_source/BUILD.bazel:30:8: Executing genrule @ios_opencv_source//:build_opencv_xcframework failed: (Exit 1): bash failed: error executing command (from target @ios_opencv_source//:build_opencv_xcframework) /bin/bash -c ... (remaining 1 argument skipped)

Use --sandbox_debug to see verbose messages from the sandbox and retain the sandbox build root for debugging
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance
Traceback (most recent call last):
  File "/private/var/tmp/_bazel_user/cabae5a58991828d2e35db74d5b63d10/sandbox/darwin-sandbox/2219/execroot/mediapipe/external/ios_opencv_source/opencv-4.5.3/platforms/apple/build_xcframework.py", line 18, in <module>
    assert get_xcode_version() >= (12, 2), \
           ^^^^^^^^^^^^^^^^^^^
  File "/private/var/tmp/_bazel_user/cabae5a58991828d2e35db74d5b63d10/external/ios_opencv_source/opencv-4.5.3/platforms/apple/cv_build_utils.py", line 40, in get_xcode_version
    ret = check_output(["xcodebuild", "-version"]).decode('utf-8')
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 466, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/subprocess.py", line 571, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['xcodebuild', '-version']' returned non-zero exit status 1.
Target //mediapipe/tasks/ios:MediaPipeTasksCommon_framework failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 265.772s, Critical Path: 84.51s
INFO: 2425 processes: 207 internal, 2218 darwin-sandbox.
FAILED: Build did NOT complete successfully

There's something that I dont quite follow in the step:

This is incorporated into the pipeline that builds the MediaPipe frameworks. You have to clone the repo and modify the the genrule in the Bazel BUILD file: thirdaprty/opencv_ios_source.BUILD. Specifically the arg cmd in the genrule .

Internally this script runs the following OpenCV Python script that builds iOS xcframeworks: https://github.com/opencv/opencv/blob/4.5.3/platforms/apple/build_xcframework.py

We exclude many modules but don't include anything other than what the OpenCV script includes by default. Not sure if there is a command line arg to include a module. You can check the source of the OpenCV script to see what can be done.

After modifying the genrule run mediapipe/tasks/ios/build_ios_framework.sh command: FRAMEWORK_NAME=MediaPipeTasksCommon MPP_BUILD_VERSION=0.1.1 'mediapipe/tasks/ios/build_ios_framework.sh'

What do I have to modify in cmd? Sorry if I might have missed some contexts here

EDIT: Opened a new issue instead