Can't use MediaPipeTasksVision with OpenCV on iOS.

[v-hogood]

Have I written custom code (as opposed to using a stock example script provided in MediaPipe)

No

OS Platform and Distribution

iOS 15.0

MediaPipe Tasks SDK version

0.10.14

Task name (e.g. Image classification, Gesture recognition etc.)

MediaPipeTasksVision

Programming Language and version (e.g. C++, Python, Java)

Swift

Describe the actual behavior

Build fails with 346 duplicate symbols.

Describe the expected behaviour

Build succeeds.

Standalone code/steps you may have used to try to get what you need

https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/image_classification/ios

Other info / Complete Logs

In the image_classification example add:
pod 'OpenCV-Dynamic-Framework'
to the Podfile.
Then try to build in Xcode and get:
duplicate symbol '_OBJC_CLASS_$_MatOfPoint3f' in:
    /Users/howardgood/Library/Developer/Xcode/DerivedData/ImageClassifier-dkioidhfgemwblbcqjglkdhicnlo/Build/Products/Debug-iphoneos/XCFrameworkIntermediates/MediaPipeTasksCommon/MediaPipeTasksCommon.framework/MediaPipeTasksCommon[1628](MatOfPoint3f.o)
    /Users/howardgood/Library/Developer/Xcode/DerivedData/ImageClassifier-dkioidhfgemwblbcqjglkdhicnlo/Build/Products/Debug-iphoneos/XCFrameworkIntermediates/OpenCV-Dynamic-Framework/opencv2.framework/opencv2[arm64][1006](MatOfPoint3f.o)
etc. etc.

[v-hogood]

MediaPipeTasksCommon xcframework should hide its public symbols like TensorFlowLite xcframework does:

https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/ios/ios.bzl#L93

[schmidt-sebastian]

Hello - unfortunately, our setup makes hiding the symbols quite difficult since we need to reference the symbols from multiple Pods (MediaPipeTasksCommon and MediaPipeTasksVisions are the main ones). Are you able to build from source and create a single static library? Then symbol hiding becomes much more feasible.

[v-hogood]

I looked and all 346 OpenCV symbols are coming from MediaPipeTasksCommon, so I guess they need to be hidden there and whatever functionality MediaPipeTasksVision etc. uses needs to be wrapped with MediaPipe functions right?

I can try building from source but I'm sure I'm not the only one who's going to want MediaPipe with OpenCV. Can you point me at build instructions or even better build scripts for iOS and Android?

[kuaashish]

Hi @schmidt-sebastian,

Can you please help out here?

Thank you!!

[uranus0206]

I have the same conflict issue which import OpenCV and MediaPipeTasksCommon xcframeworks in my project. For build MediapipeTasks xcframeworks. I just simply build from source by "build_ios_framework.sh" and commands as below:

FRAMEWORK_NAME=MediaPipeTasksCommon ./mediapipe/tasks/ios/build_ios_framework.sh FRAMEWORK_NAME=MediaPipeTasksVision ./mediapipe/tasks/ios/build_ios_framework.sh

There maybe some issue with cmake version which i issued before: https://github.com/google-ai-edge/mediapipe/issues/5491

[chenweisomebody126]

@uranus0206 Have you successfully imported both OpenCV and MediaPipeTasksCommon? I mean, is there a way to use both MediaPipeTasksVision and OpenCV? A question to @schmidt-sebastian @v-hogood

[uranus0206]

Hi @chenweisomebody126 Maybe you can try to use the same version which MediaPipe used internally. When we built mediapipe from source code, we found it use opencv 4.5.3.

Built the same version of opencv from source code. And import both xcframework in to our project without conflicts.

[chenweisomebody126]

@uranus0206 Just want to understand you what you said. You were saying:

• You built mediapipe from source code, which uses its internal opencv 4.5.3
• You were suggesting I build same version of external opencv from source code and then built mediapipe from source code that uses internal opencv 4.5.3 Maybe this mediapipe framework(including internal opencv) and external opencv framework won't have conflict.

Right?

[uranus0206]

@uranus0206 Just want to understand you what you said. You were saying:

• You built mediapipe from source code, which uses its internal opencv 4.5.3
• You were suggesting I build same version of external opencv from source code and then built mediapipe from source code that uses internal opencv 4.5.3 Maybe this mediapipe framework(including internal opencv) and external opencv framework won't have conflict.

Right?

Yes, we built mediapipe and opencv from source and use both xcframworks in out main project without conflicts.

[kslbdev]

Is there a workaround?