MacOS ARM build is broken

[tjhei]

I need to investigate why the arm build on MacOS currently fails when installing Trilinos. This might be due to #297 ... The error message is ld: symbol(s) not found for architecture arm64 collect2: error: ld returned 1 exit status

[Francyrad]

Thank you, I'll follow the resolution problem

[tjhei]

Going back to v9.4.0-r2 fixes the issue:

git checkout v9.4.0-r2

It is not because of the update to 13.2, but related to explicit instantiations, I think.

[Francyrad]

Dear tkhei

This solution doesn't work. I installed trilinos positively. However, i cannot install dealii now! This is not the only problem, i cannot install many packages that i need to run the program that i want to install (Aspect). The aspect's developers recommend:

deal.II 9.3 or newer configured with: MPI, Trilinos, p4est (required) BLAS/LAPACK, zlib (strongly recommended) HDF5 (optional) optional: PerpleX, LIBDAP, NETCDF

I could not install blas, lapack and dealii with 9.4

Do you have any solution for this? For example, installing trilinos with 9.4 and other packages with newer versions, i don't even know if this is possible...

Thank you for your work, I’m waiting for new news

[tjhei]

This solution doesn't work. I installed trilinos positively. However, i cannot install dealii now!

I can confirm that deal.II and ASPECT work for me with my suggestion. Please try removing everything from the output directory and run again. If it still fails, send the precise error message.

i cannot install many packages that i need to run the program that i want to install

You just need "p4est trilinos deal.II" from candi as described here: https://github.com/geodynamics/aspect/wiki/Installation-on-ARM-OSX

[Francyrad]

It cannot build dealii now, i attach you the error...

[dealii_9.4_error.txt](https://github.com/dealii/candi/files/9902606/dealii_9.4_error.txt)

I apologize for the inconvenience. I deleted everything the installer tried to install previously... I hope you can solve

[tjhei]

The error you are seeing is due to hdf5. Try installing without hdf5 for now.

[tjhei]

Trilinos 12.4 and 13.2 are broken with explicit instantiations enabled on ARM.

hdf5 causes a problem when compiling deal.II.

[NikkiGwh]

hi @tjhei I am facing exactly the same problem with my new M1 Pro mac (I starting to regret my choice..) After following your instructions to switch to branch "v9.4.0-r2" and invoking the installation script with: sudo ./candi.sh -j 8 --packages="p4est trilinos dealii" (without hdf5) I still get following errors: It seems like there are some issues with "boost"..

Anyway, if you have a solution for this problem let us know.

kind regards Niklas

[Francyrad]

@nikkighw I solved in this way months ago (with 9.4):

git clone https://github.com/dealii/candi cd candi ./candi.sh -j 8 --packages="p4est trilinos dealii"

I installed hdf5 in other ways.

[tjhei]

@NikkiGwh : Can you set BUILD_EXAMPLES=OFF in candi.cfg?

[NikkiGwh]

@tjhei thank you for your quick response !

the build finished successfully with turning BUILD_EXAMPLES=OFF. I used branch [v9.4.0-r2] (to avoid the trilions 13.2 error) and I didn't install hdf5. Since the example were not able to build.. do you know which features I can't use with my current installation?

@Francyrad So you were able to build it from the normal main branch without switching to [v9.4.0-r2] ? How did you fix the trillions 13.2 error then?

And could you tell me how you installed hdf5 separately (using brew or build manually etc.) and what kind of configuration you had to do so that candi was able to find this installation (cmake configuration or adding stuff to path variable etc.)

[Francyrad]

Maybe I explained it wrong. I installed version 9.4, which is enough to use the program of my interest. As for hdf5, I installed it with "brew install hdf5". I don’t remember doing other things in particular, since I solved the problem as Thjei suggested.

[KiralyAgi]

Hi Timo (@tjhei)!

As I mentioned in issue #316, I am now facing problems with the trilinos installation too. I looked over the suggestions here, but none of the problems seems the same as mine (see below) and none of the solution helps.

Here is the error: ld: warning: ignoring file /opt/homebrew/Cellar/gcc/12.2.0/lib/gcc/current/gcc/aarch64-apple-darwin22/12/libgcc.a, building for macOS-x86_64 but attempting to link with file built for macOS-arm64 Undefined symbols for architecture x86_64: "_VerifyFortran", referenced from: _main in main.c.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make[3]: *** [VerifyFortranC] Error 1 make[2]: *** [CMakeFiles/VerifyFortranC.dir/all] Error 2 make[1]: *** [CMakeFiles/VerifyFortranC.dir/rule] Error 2 make: *** [VerifyFortranC] Error 2

Is there a way to get around this? I use the mpicc, mpicxx and mpifort from my /usr/local/bin folder (i.e. not the homebrew installations), because that was working for the hdf5 and p4est installation. Update: I get the same error also with the homebrew link to mpi.

[Francyrad]

What version of MacOS do you have? If you upgraded to MacOS Ventura you may face problems, because it doesn't full support Fortran

[KiralyAgi]

Indeed, I updated to Ventura 13.1. Is there a way to get around this problem?

[Francyrad]

The problem may be MacOS Ventura, but I'm not sure. Can you please do a make clean, make again any then post there the full error with a .txt?

If the problem is Ventura, you should downgrade in a non easy process. The best would be to use Time Machine if you did a backup

[KiralyAgi]

CMakeError.log Here is the error file. I wasn't sure how(&where) to do make clean, but after clearing the tmp folder, I tried again to install trilinos via candi, with the same error.

[tjhei]

Agi: Are you running in a native terminal? What does machine return when you run it in the terminal?

Also, what does git show -q and git diff show?

[KiralyAgi]

I think so: (base) kiraly@Agness-MacBook-Pro ~ % machine arm64e (base) kiraly@Agness-MacBook-Pro ~ % git show -q fatal: not a git repository (or any of the parent directories): .git (base) kiraly@Agness-MacBook-Pro ~ % cd /Applications/candi (base) kiraly@Agness-MacBook-Pro candi % machine arm64e (base) kiraly@Agness-MacBook-Pro candi % git show -q commit 438b4a17de34a66776f69765f84dacb0e6913764 (HEAD, tag: v9.4.0-r2) Author: Timo Heister [email protected] Date: Sat Jul 2 13:08:58 2022 -0400

version 9.4.0-r2

(base) kiraly@Agness-MacBook-Pro candi % git diff diff --git a/candi.cfg b/candi.cfg index 2638bcd..487ad5b 100644 --- a/candi.cfg +++ b/candi.cfg @@ -37,7 +37,7 @@ NATIVE_OPTIMIZATIONS=OFF USE_64_BIT_INDICES=OFF

Option {ON|OFF}: Enable building of dealii examples?

-BUILD_EXAMPLES=ON +BUILD_EXAMPLES=OFF

Option {ON|OFF}: Unset CXX and set the compiler as MPI_CXX_COMPILER when configuring deal.II

//NOTE: I also tried the installation on the dealii-9.4 branch with the same results.

[bobmyhill]

Hi @KiralyAgi,

I've just successfully installed deal.II and ASPECT on the Apple M1 Pro, Ventura 13.1 (22C65)

• Use this version of candi: https://github.com/dealii/candi/releases/tag/v9.4.0-r2 (this exact commit)
• Use these instructions: https://github.com/dealii/dealii/wiki/Apple-ARM-M1-OSX (including update of software with homebrew)
• No need to change BUILD_EXAMPLES=ON
• Follow the instructions at the end of the output from candi
• ASPECT then compiled from source

Result:

---

-- This is ASPECT, the Advanced Solver for Problems in Earth's ConvecTion. -- . version 2.5.0-pre (field_types_for_elasticity_2, 9234dde4e) -- . using deal.II 9.4.0 -- . with 32 bit indices and vectorization level 0 (64 bits) -- . using Trilinos 12.18.1 -- . using p4est 2.3.2 -- . running in DEBUG mode -- . running with 8 MPI processes

[KiralyAgi]

Hi Bob!

I tried many times and the hdf5 and p4est installation goes through if I export mpicc, mpicxx, FC and FF from my usr/local/bin. But whatever I do I always get problem with trilinos, which seems to try building for architecture x86_64 instead of arm64e.

[bobmyhill]

Hi @KiralyAgi

Ok, a few ideas:

• Did you delete the trilinos-x and tmp directories inside the dealii-candi directory before retrying your installation?
• For me, mpicc, mpicxx and mpifort are linked from /opt/homebrew/bin/. I think your files in usr/local/bin are probably symlinks though (maybe worth checking).
• For what it's worth, my bashrc exports are:

  export CC=mpicc export CXX=mpicxx export FC=mpifort export FF=mpifort export OMPI_CC=gcc export OMPI_CXX=g++ export OMPI_FC=gfortran

• Maybe worth also checking which compilers you're using (see below for mine).
• If you haven't updated Command Line Tools recently, it might be worth doing that (tedious though it is): https://stackoverflow.com/a/47725718/6272561

$ mpicc -v Apple clang version 14.0.0 (clang-1400.0.29.202) Target: arm64-apple-darwin22.2.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

$ mpicxx -v Apple clang version 14.0.0 (clang-1400.0.29.202) Target: arm64-apple-darwin22.2.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

$ mpifort -v Using built-in specs. COLLECT_GCC=/opt/homebrew/bin/gfortran COLLECT_LTO_WRAPPER=/opt/homebrew/Cellar/gcc/12.2.0/bin/../libexec/gcc/aarch64-apple-darwin21/12/lto-wrapper Target: aarch64-apple-darwin21 Configured with: ../configure --prefix=/opt/homebrew/opt/gcc --libdir=/opt/homebrew/opt/gcc/lib/gcc/current --disable-nls --enable-checking=release --with-gcc-major-version-only --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-12 --with-gmp=/opt/homebrew/opt/gmp --with-mpfr=/opt/homebrew/opt/mpfr --with-mpc=/opt/homebrew/opt/libmpc --with-isl=/opt/homebrew/opt/isl --with-zstd=/opt/homebrew/opt/zstd --with-pkgversion='Homebrew GCC 12.2.0' --with-bugurl=https://github.com/Homebrew/homebrew-core/issues --with-system-zlib --build=aarch64-apple-darwin21 --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 12.2.0 (Homebrew GCC 12.2.0)

[emcgowen]

Hi @bobmyhill

This method with branch "v9.4.0-r2" works for me to build deal.II but not ASPECT. I used the compiler exports requested by the messages when you start candi.sh.

I've just successfully installed deal.II and ASPECT on the Apple M1 Pro, Ventura 13.1 (22C65)

• Use this version of candi: https://github.com/dealii/candi/releases/tag/v9.4.0-r2 (this exact commit)
• Use these instructions: https://github.com/dealii/dealii/wiki/Apple-ARM-M1-OSX (including update of software with homebrew)
• No need to change BUILD_EXAMPLES=ON
• Follow the instructions at the end of the output from candi
• ASPECT then compiled from source

Using this method I have been able to build Deal.II on an M1 Mac running Ventura 13.2, but step-32 does not run successfully (it aborts with a timeout).

Also Aspect fails to build with the same compiler exports as I used to run candi.sh: CC=mpicc CXX=mpicxx FC=mpifort FF=mpifort OMPI_CXX=g++-11 OMPI_FC=gfortran-11

mpicc -v Apple clang version 14.0.0 (clang-1400.0.29.202) Target: arm64-apple-darwin22.3.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

mpicxx -v Using built-in specs. COLLECT_GCC=/opt/homebrew/bin/g++-11 COLLECT_LTO_WRAPPER=/opt/homebrew/Cellar/gcc@11/11.3.0/bin/../libexec/gcc/aarch64-apple-darwin22/11/lto-wrapper Target: aarch64-apple-darwin22 Configured with: ../configure --prefix=/opt/homebrew/opt/gcc@11 --libdir=/opt/homebrew/opt/gcc@11/lib/gcc/11 --disable-nls --enable-checking=release --with-gcc-major-version-only --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-11 --with-gmp=/opt/homebrew/opt/gmp --with-mpfr=/opt/homebrew/opt/mpfr --with-mpc=/opt/homebrew/opt/libmpc --with-isl=/opt/homebrew/opt/isl --with-zstd=/opt/homebrew/opt/zstd --with-pkgversion='Homebrew GCC 11.3.0' --with-bugurl=https://github.com/Homebrew/homebrew-core/issues --build=aarch64-apple-darwin22 --with-system-zlib --with-sysroot=/Library/Developer/CommandLineTools/SDKs/MacOSX13.sdk Thread model: posix Supported LTO compression algorithms: zlib zstd gcc version 11.3.0 (Homebrew GCC 11.3.0)

After compiling the WorldBuilder files, the Unity files give compiler errors that gcc-11 does not recognize the flag -Xarch-arm64:

[58%] Building CXX object CMakeFiles/aspect.dir/contrib/world_builder/source/wrapper_c.cc.o [ 64%] Building CXX object CMakeFiles/aspect.dir/contrib/world_builder/source/wrapper_cpp.cc.o [ 64%] Building CXX object CMakeFiles/aspect.dir/contrib/world_builder/source/parameters.cc.o [ 64%] Building CXX object CMakeFiles/aspect.dir/Unity/unity_39_cxx.cxx.o [ 64%] Building CXX object CMakeFiles/aspect.dir/Unity/unity_38_cxx.cxx.o [ 64%] Building CXX object CMakeFiles/aspect.dir/Unity/unity_36_cxx.cxx.o [ 64%] Building CXX object CMakeFiles/aspect.dir/Unity/unity_42_cxx.cxx.o [ 64%] Building CXX object CMakeFiles/aspect.dir/Unity/unity_35_cxx.cxx.o [ 70%] Building CXX object CMakeFiles/aspect.dir/Unity/unity_37_cxx.cxx.o [ 70%] Building CXX object CMakeFiles/aspect.dir/Unity/unity_41_cxx.cxx.o [ 70%] Building CXX object CMakeFiles/aspect.dir/Unity/unity_40_cxx.cxx.o g++-11: error: unrecognized command-line option '-Xarch_arm64' g++-11: error: unrecognized command-line option '-Xarch_arm64' g++-11: error: unrecognized command-line option '-Xarch_arm64' g++-11: error: unrecognized command-line option '-Xarch_arm64' g++-11: error: unrecognized command-line option '-Xarch_arm64' make[2]: *** [CMakeFiles/aspect.dir/Unity/unity_40_cxx.cxx.o] Error 1 make[2]: *** Waiting for unfinished jobs.... make[2]: *** [CMakeFiles/aspect.dir/Unity/unity_41_cxx.cxx.o] Error 1

Perhaps the wrong compiler is being invoked for these files? The mpicxx compiler is g++-11, and I've tried passing the Apple Clang compiler for mpicxx, but then I get syntax errors from the worldbuilder files.

Also, the ASPECT source was checked out with: git clone https://github.com/geodynamics/aspect.git Is this a stable version or a development version?

In case this was a problem only with top-of-trunk, I tried building ASPECT with the aspect-2.4.0.tar release and got the exact same compiler errors as before.

Thanks

[bobmyhill]

Hi @emcgowen

Just to check my instructions still work, I updated my M1 Macbook to Ventura 13.2, deleted and reinstalled xcode CLT (https://stackoverflow.com/a/47804075/6272561) and then deleted and reinstalled deal.II and ASPECT as per the instructions in my original post. Everything still works fine.

Some output before I reinstalled deal.II:

$ pkgutil --pkg-info=com.apple.pkg.CLTools_Executables package-id: com.apple.pkg.CLTools_Executables version: 14.2.0.0.1.1668646533

$mpicxx -v Apple clang version 14.0.0 (clang-1400.0.29.202)

From what you've written, the mpicxx alias does not point to clang, as recommended in the instructions linked to in my original post (https://github.com/dealii/dealii/wiki/Apple-ARM-M1-OSX). You should make sure that mpicxx points to the right compiler, then delete your deal.II installation (probably ~/dealii-candi/ if you didn't change the destination in candi.cfg), and try again.

[emcgowen]

This solved it for me, and now I have a working Deal II and an ASPECT which prints out:

---

-- This is ASPECT, the Advanced Solver for Problems in Earth's ConvecTion. -- . version 2.5.0-pre (main, 2b14e1b8e) -- . using deal.II 9.4.0 -- . with 32 bit indices and vectorization level 0 (64 bits) -- . using Trilinos 12.18.1 -- . using p4est 2.3.2 -- . running in DEBUG mode -- . running with 1 MPI process ——————————————————————————————————————

For future reference, the instructions about how to use homebrew need to be corrected. By default

brew install openmpi

installs mpicc = gcc and mpicxx = gcc. To keep the native clang compiler you must do

brew CC=clang install openmpi

This causes brew to recompile openmpi using the clang compiler.

Also, candi.sh prints out misleading instructions about exporting compiler environment variables:

export CC=mpicc; export CXX=mpicxx, FC=mpifort, FF=mpifort
export OMPI_CC=gcc-11, export OMPI_CXX=g++-11, export OMPI_FC=gfortran+11

DO NOT export the OMPI_ environment variables unless you intend to build with the GNU C compiler!

And finally, a nit. When testing Deal II with the step-32 test, I had to do

export LC_ALL=C

to get the test to run.

Thanks!

Ellen McGowen

On Feb 5, 2023, at 4:35 AM, Bob Myhill @.***> wrote:

Hi @emcgowen https://github.com/emcgowen Just to check my instructions still work, I updated my M1 Macbook to Ventura 13.2, deleted and reinstalled xcode CLT (https://stackoverflow.com/a/47804075/6272561) and then deleted and reinstalled deal.II and ASPECT as per the instructions in my original post. Everything still works fine.

Some output before I reinstalled deal.II:

$ pkgutil --pkg-info=com.apple.pkg.CLTools_Executables package-id: com.apple.pkg.CLTools_Executables version: 14.2.0.0.1.1668646533

$mpicxx -v Apple clang version 14.0.0 (clang-1400.0.29.202)

From what you've written, the mpicxx alias does not point to clang, as recommended in the instructions linked to in my original post (https://github.com/dealii/dealii/wiki/Apple-ARM-M1-OSX). You should make sure that mpicxx points to the right compiler, then delete your deal.II installation (probably ~/dealii-candi/ if you didn't change the destination in candi.cfg), and try again.

— Reply to this email directly, view it on GitHub https://github.com/dealii/candi/issues/309#issuecomment-1417712964, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABTDMVLULMJYZMHQSLEXQC3WV6NAZANCNFSM6AAAAAARPBZ2ZA. You are receiving this because you were mentioned.

[tjhei]

brew --cc=clang install openmpi

Thank you, I updated the instructions at https://github.com/dealii/dealii/wiki/Apple-ARM-M1-OSX

[bobmyhill]

Glad it worked! Thank you for the additional information, that's very useful. And thanks again @tjhei for all your work on this problem.

[Francyrad]

Hello. I needed to reinstall Dealii to make aspect workwith Ventura 13.1. So i downloaded the version you said above. Then ./candi.sh -j 8 --packages="p4est trilinos dealii" It install p4est and trillnos, but not dealii. This is the error. How can i solve? Thank you for your support

[tjhei]

You are not showing the error message, can you post errors higher up?

[Francyrad]

candierror2.txt Sure!