[Work in progress] OpenCV recipe

[meow464]

I'm opening this issue to document the progress on creating an OpenCV recipe. Hopefully others will help me in this task and this issue will be of help to others tackling similar problems. Feel free to give any kind of suggestions and/or brainstorm. I use the same nick on discord.

There are 3 large steps:

1. Get opencv to work on iOS;
2. Get opencv-python to compile to iOS.
3. Create one or more new recipe classes to automate the work above.

1. Get opencv to work on iOS

This is easy since opencv already works on iOS. The only problem is that kivy copied its iOS camera code from opencv itself and thus there are some symbol conflicts. Simply rename CaptureDelegate to KivyCaptureDelegate in core/camera/camera_avfoundation_implem.m then follow the opencv iOS tutorial.

2. Get opencv-python to compile to iOS

This is the hard part. We need to compile opencv-python and link it against kivy's python. Here are some approaches that could work:

2.1 Do it as our forefathers did on the desktops of ol'

A good first step is to take a look at the android recipe. On the patch file we can get an idea of how to disable the checks that disable opencv-python compilation. On the init.py we can get an idea of the cmake parameters required. Anything you get from here should probably be tested on opencv/platforms/ios/build_framework.py This approach should compile opencv-python together with opencv, the same way it's done on desktops. Unfortunately I'm still not able to make it work.

2.2 Copy the CI setup.py

Another possible approach is to look at the CI for opencv-python setup.py. It also gives some ideas on how to link it against kivy's python.

2.3 probably the hardest way

Yet another approach is to attack the python binding's directly. I haven't tried this yet.

3. Create one or more new recipe classes to automate the work above

We will need at least a class XCodeFrameworkRecipe, that's a class that builds a framework and adds it our project, this is very easy to do. Depending on the approach that succeeds in step 2 we will also need something on the lines of class CompiledComponentsRecipe, not particularly hard either.

[meow464]

For reference this is my modified build_framework.py. All differences should be on the getCMakeArgs function. Please note that the paths are hardcoded, you will have to change them to test on your machine.

[xu-xionglong]

For reference this is my modified build_framework.py. All differences should be on the getCMakeArgs function. Please note that the paths are hardcoded, you will have to change them to test on your machine.

Hello, does this build script now work for building python module on iOS? Or it still has some problems to solved?

[meow464]

It does not work yet. I plan on getting back to it soon™. In the meantime you could use C/C++ and a custom Cython recipe to interact with CPython. There are people in the Discord who have done this.

[Neizvestnyj]

Hi, are there any changes or is the recipe creation stalled?

[meow464]

@Neizvestnyj I'm currently working on Fortran -> Lapack -> Numpy/Scipy. I think I stopped hitting super hard problems and might have time for OpenCV soon™. In any case, OpenCV depends on Numpy and it is still not clear to me if it's usable without lapack acceleration.

In the meantime you could write the OpenCV part in C or C++ and call it form python. Just make a new C/C++ file in XCode, call what you need on OpenCV and expose a C interface to python with extern C. Python has very good interoperability with C you can just import the C interface.

[Neizvestnyj]

I understand, I just don't want to overload the code. If you make a recipe, I will be very happy

[simonmahdavi]

@meow464 Thanks for your work so far! Do you have any examples of someone that have written the openCV part in C/C++?

[meow464]

@simonmahdavi Someone on the kivy discord did it but I don't remember who.

[prasadmadanayake]

@meow464 I created a opencv recipe disabling some opencv features which i mainly use for camara and image processing. recipe is still not refined but you can use it in ios if it match your requirement. also you need to remove libjpeg from project if you are using opencv since libjpeg-turbo which is used by opencv will have conflicts with it. I will make the PR once it completed so untill then you can use this fork https://github.com/prasadmadanayake/kivy-ios

[meow464]

@prasadmadanayake That's great, I will take a look!

[lukebelz]

@prasadmadanayake This looks very promising but I get a "18 duplicate symbols for architecture arm64" error on build. If I build numpy by itself, I can import it and no errors. But when I build opencv, and create a new xcode project, and run it, that's what I get.

I don't have libjpeg install btw.

[lukebelz]

@prasadmadanayake This looks very promising but I get a "18 duplicate symbols for architecture arm64" error on build. If I build numpy by itself, I can import it and no errors. But when I build opencv, and create a new xcode project, and run it, that's what I get.

I don't have libjpeg install btw.

I actually just fix it by removing the libzlib.a archive from the project.

I can load images just fine into a numpy array. But imwrite() gives me a blank jpeg file. The data before this is fine. I had the same issue using pillow to write an image. Is there some permission or something that needs to be enabled in order to write images to the user_data_dir?

When I run kivy on my MPB, both pillow and opencv write the image just fine. Something is different about running it with kivy-ios or on an ipad.

[tcaduser]

It looks like opencv already provides iOS builds.

https://docs.opencv.org/4.x/d3/dc9/tutorial_table_of_content_ios.html https://github.com/opencv/opencv/releases/download/4.8.0/opencv-4.8.0-ios-framework.zip