[RFC] Re-add support for macOS

[philippremy]

[WIP] Project support for macOS

Well, I decided that this might be ready for an RFC now. It’s still a draft because I don’t yet have a solid solution for compiling the Boost libraries on macOS reliably — currently, only AppleClang is supported due to issues with the Boost CMake build system and an LLVM upstream issue. Once that is solved, there are no outstanding no-gos from my side.

I know that these look like some heavy changes, but I believe that I did not break any Windows/Linux infrastructure. I tried my best to do CI on both Windows and Linux, and at least with the official GitHub CI, they appeared to work. Compiling embedded dependencies on Linux works as expected, if you don't attempt to build an embedded Assimp - but that issue was already present before (on recent GCCs, one header is missing a #include <cstdint>, which is a one-line fix).

Overview of changes

Each commit has its own description on what changed and why that change is required in my opinion. But as a quick overview:

• OFA: The whole OptimizeForArchitecture infrastructure has been reworked. Previously, only x86_64 was properly supported, now there is very solid arm/arm64, expanded x86_64 and rudimentary ppc support.
• Dependencies: Various embedded dependency version liftings - all changes here appear ABI and API compatible, but mainly solve build issues on macOS present in the older versions (see the individual commit messages). This does not raise the minimum versions required globally, so the current minimum dependency rules still apply.
• macOS bundler: I wrote a small Python tool for creating a standalone bundle on macOS. This allows to create fully redistributable packages that do not depend on libraries other than those provided by macOS itself.
• Configuration variables: When building embedded dependencies, only a small subset of CMake variables was passed through to the actual AliceVision subproject (/src). Now the user can overwrite all variables properly by passing them on the CMake CLI (backwards-comparible).
• Remove BLAS/LAPACK and Suitesparse on macOS: Apple provides Accelerate.framework, which Ceres can use instead of Suitesparse. This lifts the requirement for a Fortran compiler on macOS and drastically reduces compilation times and bundle size. When running the unit tests I did not notice any performance differences.
• CUDA: As CUDA is completely unsupported on Apple platforms, all related functionality is properly guarded behind conditional checks in the build infrastructure.
• Cross-compiling: As cross-compiling on macOS is usually a matter of passing -arch <ARCH> to the compiler, the project fully supports cross-compiling from arm64 to x86_64 on macOS - including all required dependencies. The target architecture can be set by specifying CMAKE_OSX_ARCHITECTURES=<arm64|x86_64> on the CMake CLI. Note that compiling universal binaries is not supported at this point, because some dependencies do not have the required logic to do this in one step.
• Build manual: A special INSTALL_macOS.md provides specific instructions on how to compile this project successfully on macOS.
• OpenMP: Using OpenMP with AppleClang is now supported - this requires that OpenMP is installed as an external dependency (enabled by default on macOS), as the Apple SDK does not ship it. Furthermore, the project now uses the unified CMake targets OpenMP::OpenMP_C and OpenMP::OpenMP_CXX respectively instead of setting the required flags manually.

[TBA] Open questions

• [ ] Reliable Boost build with cross-compiling capabilities
• [ ] Remove PCL dependency because it is unused (see #2012)

Drawbacks

Because AliceVision has so many dependencies - some with very specific configurations - I think we cannot officially support building the project with package-manager provided dependencies. I just stumbled from roadblock to roadblock when attempting to standardize the required dependencies for the different package managers. So currently the only option is to build the project with ALICEVISION_BUILD_DEPENDENCIES=ON. However, this matches the recommendation for Linux, so I think this should be fine. If someone wants to attempt to build it with Homebrew/MacPorts/Nix anyway, that might or might not work. But at least we don't have the obligation to ensure compatibility for these cases.

There is no CI yet. I decided against CI, because it currently requires to rebuild all dependencies over and over again - the GitHub runners usually only have 3 threads available, so this is quite slow. I tested pre-built dependencies, but that currently does not work because there are some dependencies hard-code absolute paths into their CMake config modules - making them not portable.

There are no unit tests yet. I ran the unit tests on my Mac, and two are currently failing:

1. acRansac_test.cpp: I think there is some undefined behavior here, which caused a segfault on macOS. See this gist for a detailed description and proposed solution.
2. fundamental10PSolver_test.cpp: The check on line 184 fails. I have no idea why and I don't know anywhere near enough to solve this. Help is much appreciated. For the results see this gist.

Future plans

This is the first step to make macOS a fully supported platform for this project again. The next steps would be to add the Apple Metal DepthMap backend and maybe generally abstract the DepthMap library to allow for multiple backends with a unified interface.

Also: Add a proper CI solution and ensure that all unit tests pass.

Feedback is appreciated!

These patches are the result of weeks of debugging, investigating and rebasing - but I am sure that there are still issues and edge cases that I completely missed. So fresh viewpoints and remarks are much appreciated. If anyone has the chance to test this out, please do so and let me know if there are any issues.

[servantftransperfect]

Sorry for the long time to answer, i'm still figuring on how to get access to a mac.

[philippremy]

No worries, I'll be busy with exams until the end of February anyways, so I won't be able to work on this full time until then. Just wanted to get this out, so at least the RFC period can start :).