Add support for M1 macs

[vieux]

Is your feature request related to a problem?

Exiv2 darwin builds are only available for x86_64, not arm64/aarch64

Describe the solution you would like

Update build scripts and CI to build on M1 macs

Describe alternatives you have considered

I tried building locally, but the binary produced are still x86_64

[clanmills]

Thank you for this request.

At the time of the 0.27.5 release, my MacBookPro with the m1 chip was generating thousands of linker visibility warnings which I didn't understand. The x86_64 build was clean. So, I only provided the x86_64 on Darwin.

I'm happy to say that Luis (@piponazo) solved the linker issue on branch main, however the fix cannot be back-ported to 0.27-maintenance because it will break the 0.27 API. If there is a 0.27.6 release, the m1 build is unlikely to be provided because or the linker warnings.

The release of binary builds is a relic from the days before CI. Team Exiv2 decided to discontinue binary releases after the 0.27 "dots" because we provide daily builds from branch main.

Can a member of Team Exiv2 confiirm/configure the macOS daily to build both x86_64 and arm64/aarch64?

[clanmills]

@vieux. I frequently (almost daily) build exiv2 on my MBP with the m1 chip. Both branches 0.27-maintenance and main build arm64 binaries:

635 rmills@rmillsm1:~/gnu/github/exiv2 $ file main/build/bin/exiv2
main/build/bin/exiv2: Mach-O 64-bit executable arm64
636 rmills@rmillsm1:~/gnu/github/exiv2 $ file 0.27-maintenance/build/bin/exiv2
0.27-maintenance/build/bin/exiv2: Mach-O 64-bit executable arm64
637 rmills@rmillsm1:~/gnu/github/exiv2 $ 

I got the MBP/m1 in February 2021 and the machine has always built arm64 binaries. I had some initial difficulties with the linker wanting to link arm64 build artefacts with x86_64 libraries in /usr/local/lib which the Apple Transfer utility had copied from my "old" Intel MBP.

The current version of Xcode is 13.3 and I have cmake 3.22.1 installed. I will be happy to screen share with you on Zoom to assist you to get up-and-running.

[piponazo]

Can a member of Team Exiv2 confiirm/configure the macOS daily to build both x86_64 and arm64/aarch64?

We are only building with x86_64 at the moment: https://github.com/Exiv2/exiv2/actions/runs/2052494056

It would be nice to add support for the arm64/aarch64 , but I am afraid there are not such machines in the Github environments at the moment: https://github.com/actions/virtual-environments/issues/2187

Therefore, I do not think we can do much about it in CI. Of course, we could add some instructions in our README in case we need to do something special for that platform.

[clanmills]

I'm not aware of anything special about building on the Apple/arm64/m1 chip, so I don't think it's necessary to change README.md for the m1.

I believe Xcode running on Intel can build arm64 and "universal" builds. A "universal" has both x86_64 and arm64 code. I'll investigate. If this is possible, I'll update README.md to discuss this.

[clanmills]

I have built x86_64 on the m1 machine. The cmake documentation is here: https://cmake.org/cmake/help/latest/variable/CMAKE_APPLE_SILICON_PROCESSOR.html#variable:CMAKE_APPLE_SILICON_PROCESSOR

_/0.27-maintenance/ $ rm -rf  build ; mkdir build ; cd build
_/0.27-maintenance/build $  cmake .. -DEXIV2_ENABLE_BMFF=1  -DCMAKE_APPLE_SILICON_PROCESSOR=x86_64
...
_/0.27-maintenance/build $ make
...
_/0.27-maintenance/build $ file /Users/rmills/gnu/github/exiv2/0.27-maintenance/build/bin/exiv2
/Users/rmills/gnu/github/exiv2/0.27-maintenance/build/bin/exiv2: Mach-O 64-bit executable x86_64
_/0.27-maintenance/build $ 

I know that I do not have x86_64 libraries on my m1, so I believe it's linking expat and zlib using macOS/SDK installed by Xcode.

I'll investigate "universal" builds later today.

[clanmills]

I have built "universal" on the m1 machine based on this discussion: https://stackoverflow.com/questions/65157483/macos-build-universal-binary-2-with-cmake

_/main $ rm -rf build ;mkdir build ; cd build
_main/build $ cmake .. -DEXIV2_BUILD_SAMPLES=1 '-DCMAKE_OSX_ARCHITECTURES=x86_64;arm64'
...
_main/build $ make
...
 _/main/build $ file bin/exiv2
bin/exiv2: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64:Mach-O 64-bit executable arm64]
bin/exiv2 (for architecture x86_64):	Mach-O 64-bit executable x86_64
bin/exiv2 (for architecture arm64):	Mach-O 64-bit executable arm64
_/main/build $ 

Observations:

1. arm64 failed on Xcode 12/Intel The arm64 build quickly died in stdint.h on my elderly Intel/MacMini using Xcode 12.1. We should see if the macOS/CI has Xcode 13 and can build "universal".

2. Deprecation warnings coming from samples/Jzon.h This has something to do with C++17. The Jzon code is elderly and no longer supported by the developer. Let's remove sample/exiv2json.cpp (and Jzon.h and Jzon.cpp) from the mix and update README-SAMPLES.md

3. Packaging and Documentation There will be packaging and documentation changes if we build "universal". I'm willing to make those changes when/if we decide to build "universal" on main/daily.

4. Fedback from @vieux Let's hear what @vieux has to say before deciding how to proceed.

5. 0.27-maintenance universal I don't think we should support this. 0.27-maintenance will soon be 4 years old. We need a plan to release 'main' and allow 0.27-maintenance to retire (and for me to finally retire as well, of course).

[clanmills]

No user feedback. I'm going to close this.

[piponazo]

Hi @clanmills , regarding this:

2. Deprecation warnings coming from samples/Jzon.h This has something to do with C++17. The Jzon code is elderly and no longer supported by the developer. Let's remove sample/exiv2json.cpp (and Jzon.h and Jzon.cpp) from the mix and update README-SAMPLES.md

At the moment exiv2json is used in some of the bash tests:

$ grep exiv2json tests/* -R
tests/bash_tests/test_issue_1054.py:    commands = [ "$exiv2json $filename1",
tests/bash_tests/test_issue_1054.py:                 "$exiv2json x $filename1",
tests/bash_tests/test_issue_1054.py:                 "$exiv2json $filename2",
tests/bash_tests/test_issue_984.py:    commands = ["$exiv2json $filename"]

But those tests are exercising the exiv2json sample app and not too much the library ... I would also vote for removing that sample app. I can create a PR for that pointing to main

[clanmills]

@piponazo Well spotted about those tests.

#984 concerns a null-ptr read in exiv2json, So when samples/exiv2json is killed, test_issue-984.py should be removed.

test_issue_1054.py is there to prevent a regression of code that recursively parses the xmp output into JSON. So, when exiv2json.cpp dies, this test should also be removed.

@vieux I'm very surprised you haven't said anything. Is the M1 building OK for you?

It would be useful to get the CI to build "universal" on branch 'main'. I will re-open this issue.

[nitishsaDire]

hey all, I tried building on mac m1 air, but it is failing again and again. Can someone please share build steps for m1 macs, that will be very helpful. Thanks in advance.

I am stuck here, on make it is giving error ld: library not found for -linih_inireader_LIBRARY-NOTFOUND

updates: I am able to create a build by following the steps: 1, mkdir build;cd build 2, cmake .. -DEXIV2_ENABLE_XMP=ON -DEXIV2_ENABLE_VIDEO=ON -DEXIV2_ENABLE_PNG=ON -DEXIV2_ENABLE_NLS=ON -DEXIV2_ENABLE_PRINTUCS2=ON -DEXIV2_ENABLE_LENSDATA=ON -DEXIV2_ENABLE_VIDEO=ON -DEXIV2_ENABLE_WEBREADY=ON -DEXIV2_ENABLE_CURL=ON -DEXIV2_ENABLE_SSH=ON -DEXIV2_ENABLE_BMFF=ON -DEXIV2_BUILD_SAMPLES=OFF -DCMAKE_BUILD_TYPE=Release 3, make I then put this created build folder on another Mac m1 device and tried to run exiv2 as follow: 1, cd to build directory 2, export DYLD_LIBRARY_PATH="$PWD/lib:$DYLD_LIBRARY_PATH" 3, ./bin/exiv2 it gives the error Library not loaded: /opt/homebrew/opt/libssh/lib/libssh.4.dylib, which I think means that the build I have created is not an m1 standalone exiv2 build, like other standalone builds available at https://exiv2.org/download.html in build section for other systems.

Any direction on how to create a standalone m1 build would be helpful. Thanks

Updates: Finally able to create a standalone exiv2 build for mac m1s. gdrive link https://drive.google.com/file/d/13zcvkOwRl4DQMGtJfH2NReQxau4scfnP/view?usp=sharing

[Avnsx]

Does anyone mind helping the python version of this project out and creating mac compatible builds?

https://github.com/LeoHsiao1/pyexiv2/issues/108

[manishjain999]

Does anyone have any updates for building / consuming EXIV2 universally for M1 and Intel machines both?