mxnet
mxnet copied to clipboard
apache
Mac-ARM build failure
Building on Mac-ARM fails with some basic errors, the first one being:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/clang/12.0.0/include/mmintrin.h:33:5: error:
use of undeclared identifier '__builtin_ia32_emms'; did you mean '__builtin_isless'?
__builtin_ia32_emms();
^
After a bit of googling I've found this document with information about the issue: https://clang.llvm.org/compatibility.html#vector_builtins
In particular, this line looks relevant:
The same issue occurs for NEON and Altivec for the ARM and PowerPC architectures respectively. For these, make sure to use the <arm_neon.h> and <altivec.h> headers.
So are Mac-ARM builds supported at all? What would be needed to add support if not?
@mseth10 has been looking into ARM build (for linux and osx alike, I think). Do you have some inputs to share here, Manu?
@matteosal we have improved the ARM builds on CI last year and in general it works very well on master. Are you using the master version?
@leezu unfortunately we are still tied to 1.x versions, and I'd like to avoid switching to 2.0 master for the time being (but we'll make the switch at some point). So we might decide to port 1.x to ARM. Given that you have ported 2.0 already, could you provide an estimate on how big of a project it is, and what needs to be done roughly? That would be very helpful.
@matteosal I'd suggest you first try if the issue is reproducible on the master branch.
could you provide an estimate on how big of a project it is, and what needs to be done roughly? That would be very helpful.
Assuming the issue is fixed on master, it should be feasible in less than a day, probably 2-3 hours, mostly spent figuring out the diff of related files between master and v1.x (identify commits that changed the ARM CI on master, look at the files changed in the same commit).
Hi @matteosal , can you share information on what platforms you are using to build and test MXNet? Are you using native or cross compilation? I have built MXNet v1.x on ubuntu x86 using cross compilation and tested on ubuntu aarch64 and it works fine.
@leezu I've managed to get master to build on Mac-ARM (with a small fix: https://github.com/apache/incubator-mxnet/pull/20072). I will now look into the changes needed by 1.x
@mseth10 I'm doing native compilation on Mac-ARM, not Linux. No testing platform because the issue is a build failure.
@matteosal Were you able to get 1.x to build on macOS? I've been trying to create a build on my M1 Mac, so that I can reference the binary locally as suggested here for DJL: https://github.com/deepjavalibrary/djl/issues/375#issuecomment-975042055
I can get the MxNet master branch to build locally, but not 1.8, 1.9, or 1.9.1RC tags. If I try to set the environment variable for DJL to pull in the MxNet 2.0 build, it'll crash (which is probably expected since 2.0 is a major new version and DJL is using MxNet 1.8).
It seems like there are two issues with the 1.x branches I can see so far:
- An issue with googletest, which I can apparently work around by copying that folder over from the master branch. The actual error is:
CMake Error at CMakeLists.txt:490 (add_subdirectory):
add_subdirectory given source
"/3rdparty/googletest/googletest"
- A compilation issue reported in CMakeError.log:
Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" failed.
Compiler: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++
Build flags:
Id flags:
The output was:
1
ld: library not found for -lc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
Check your config.cmake file, make sure the USE_SSE/USE_F16C configuration is OFF.
fwiw, was able to get the latest version (1.9.1) built with Homebrew with -DUSE_SSE=OFF. Mac ARM users should now be able to install a precompiled build with:
brew install mxnet
https://github.com/Homebrew/homebrew-core/pull/111872
