WarpX
WarpX copied to clipboard
ECP-WarpX
problems building on os X stddef.h and math.h not found
I'm using a M1 mac with most packages installed via brew (including python).
I'm trying to follow the cmake instructions from https://warpx.readthedocs.io/en/latest/install/users.html.
I tried this both on the 24.08 tag and the latest git commit (3dda26f)
Creating the build files seems to work:
arun@Aruns-MacBook-Pro-2:~/src/WarpX((24.08))>cmake -S . -B build
-- The C compiler identification is AppleClang 15.0.0.15000309
-- The CXX compiler identification is AppleClang 15.0.0.15000309
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Could NOT find CCache
-- Downloading AMReX ...
-- AMReX repository: https://github.com/AMReX-Codes/amrex.git (24.08)
CMake Warning (dev) at /opt/homebrew/Cellar/cmake/3.30.2/share/cmake/Modules/FetchContent.cmake:1953 (message):
Calling FetchContent_Populate(fetchedamrex) is deprecated, call
FetchContent_MakeAvailable(fetchedamrex) instead. Policy CMP0169 can be
set to OLD to allow FetchContent_Populate(fetchedamrex) to be called
directly for now, but the ability to call it with declared details will be
removed completely in a future version.
Call Stack (most recent call first):
cmake/dependencies/AMReX.cmake:151 (FetchContent_Populate)
cmake/dependencies/AMReX.cmake:280 (find_amrex)
CMakeLists.txt:209 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
-- CMake version: 3.30.2
-- AMReX installation directory: /usr/local
-- Build type set by user to 'Release'.
-- Building AMReX with AMReX_SPACEDIM = 3
-- Configuring AMReX with the following options enabled:
-- AMReX_PRECISION = DOUBLE
-- AMReX_MPI
-- AMReX_MPI_THREAD_MULTIPLE
-- AMReX_OMP
-- AMReX_LINEAR_SOLVERS
-- AMReX_PARTICLES
-- AMReX_PARTICLES_PRECISION = DOUBLE
-- AMReX_PIC
-- AMReX_EXPORT_DYNAMIC
-- AMReX_TINY_PROFILE
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Found MPI_C: /opt/homebrew/Cellar/open-mpi/5.0.3_1/lib/libmpi.dylib (found version "3.1")
-- Found MPI_CXX: /opt/homebrew/Cellar/open-mpi/5.0.3_1/lib/libmpi.dylib (found version "3.1")
-- Found MPI: TRUE (found version "3.1") found components: C CXX
-- Found OpenMP_CXX: -Xclang -fopenmp (found version "5.0")
-- Found OpenMP: TRUE (found version "5.0") found components: CXX
-- AMReX configuration summary:
-- Build type = Release
-- Install directory = /usr/local
-- C++ compiler = /Library/Developer/CommandLineTools/usr/bin/c++
-- C++ defines = -DAMREX_SPACEDIM=3
-- C++ flags = -O3 -DNDEBUG -Xclang -fopenmp
-- C++ include paths = -I/Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/Base -I/Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/Base/Parser -I/Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/Boundary -I/Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/AmrCore -I/Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/LinearSolvers -I/Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/LinearSolvers/MLMG -I/Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/LinearSolvers/OpenBC -I/Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/Particle -I/opt/homebrew/Cellar/open-mpi/5.0.3_1/include -I/opt/homebrew/include
-- Link line = /opt/homebrew/Cellar/open-mpi/5.0.3_1/lib/libmpi.dylib /opt/homebrew/lib/libomp.dylib
-- AMReX: Using version '24.08' (24.08)
-- Downloading PICSAR ...
-- PICSAR repository: https://github.com/ECP-WarpX/picsar.git (aa54e985398c1d575abc7e6737cdbc660a13765f)
CMake Warning (dev) at /opt/homebrew/Cellar/cmake/3.30.2/share/cmake/Modules/FetchContent.cmake:1953 (message):
Calling FetchContent_Populate(fetchedpicsar) is deprecated, call
FetchContent_MakeAvailable(fetchedpicsar) instead. Policy CMP0169 can be
set to OLD to allow FetchContent_Populate(fetchedpicsar) to be called
directly for now, but the ability to call it with declared details will be
removed completely in a future version.
Call Stack (most recent call first):
cmake/dependencies/PICSAR.cmake:63 (FetchContent_Populate)
cmake/dependencies/PICSAR.cmake:125 (find_picsar)
CMakeLists.txt:219 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Downloading openPMD-api ...
-- openPMD-api repository: https://github.com/openPMD/openPMD-api.git (0.15.2)
CMake Warning (dev) at /opt/homebrew/Cellar/cmake/3.30.2/share/cmake/Modules/FetchContent.cmake:1953 (message):
Calling FetchContent_Populate(fetchedopenpmd) is deprecated, call
FetchContent_MakeAvailable(fetchedopenpmd) instead. Policy CMP0169 can be
set to OLD to allow FetchContent_Populate(fetchedopenpmd) to be called
directly for now, but the ability to call it with declared details will be
removed completely in a future version.
Call Stack (most recent call first):
cmake/dependencies/openPMD.cmake:38 (FetchContent_Populate)
cmake/dependencies/openPMD.cmake:100 (find_openpmd)
CMakeLists.txt:223 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Found MPI: TRUE (found version "3.1") found components: CXX
-- Using the single-header code from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/share/openPMD/thirdParty/json/single_include/
-- nlohmann-json: Using INTERNAL version '3.9.1'
-- toml11: Using INTERNAL version '3.7.1'
-- Found HDF5: /opt/homebrew/Cellar/hdf5-mpi/1.14.3/lib/libhdf5.dylib;/opt/homebrew/opt/libaec/lib/libsz.dylib;/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/lib/libz.tbd;/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/lib/libdl.tbd;/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/lib/libm.tbd (found version "1.14.3") found components: C
-- Found MPI: TRUE (found version "3.1")
-- Found ADIOS2: /opt/homebrew/lib/cmake/adios2/adios2-config.cmake (found suitable version "2.10.1", minimum required is "2.7.0") found components: C CXX MPI
openPMD build configuration:
library Version: 0.15.2
openPMD Standard: 1.1.0
C++ Compiler: AppleClang 15.0.0.15000309
/Library/Developer/CommandLineTools/usr/bin/c++
Installation: OFF
Build Type: Release
Library: static
CLI Tools: OFF
Examples: OFF
Testing: OFF
Invasive Tests: OFF
Internal VERIFY: ON
Build Options:
MPI: ON
HDF5: ON
ADIOS1: OFF
ADIOS2: ON
PYTHON: OFF
CUDA_EXAMPLES: OFF
-- Found Git: /opt/homebrew/bin/git (found version "2.46.0")
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR
-- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
WarpX build configuration:
Version: 24.08 (24.08)
C++ Compiler: AppleClang 15.0.0.15000309
/Library/Developer/CommandLineTools/usr/bin/c++
Installation prefix: /usr/local
bin: bin
lib: lib
include: include
cmake: lib/cmake/WarpX
Build type: Release
Build options:
APP: ON
ASCENT: OFF
COMPUTE: OMP
DIMS: 3
Embedded Boundary: OFF
IPO/LTO: OFF
LIB: ON (static)
MPI: ON
PARTICLE PRECISION: DOUBLE
PRECISION: DOUBLE
FFT Solvers: OFF
heFFTe: OFF
PYTHON: OFF
OPENPMD: ON
QED: ON
QED table generation: OFF
QED tools: OFF
SENSEI: OFF
-- Configuring done (13.8s)
-- Generating done (0.2s)
-- Build files have been written to: /Users/arun/src/WarpX/build
But when I try to compile it, I run into errors:
arun@Aruns-MacBook-Pro-2:~/src/WarpX((24.08))>cmake --build build -j 4
[ 0%] Building CXX object _deps/fetchedopenpmd-build/CMakeFiles/openPMD.dir/src/config.cpp.o
[ 1%] Building CXX object _deps/fetchedopenpmd-build/CMakeFiles/openPMD.dir/src/ChunkInfo.cpp.o
[ 1%] Building CXX object CMakeFiles/buildInfoapp_3d.dir/app_3d/AMReX_buildInfo.cpp.o
[ 1%] Building CXX object _deps/fetchedamrex-build/Src/CMakeFiles/amrex_3d.dir/Base/AMReX_BlockMutex.cpp.o
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/src/ChunkInfo.cpp:21:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/include/openPMD/ChunkInfo.hpp:23:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/include/openPMD/Dataset.hpp:23:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/include/openPMD/Datatype.hpp:23:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/include/openPMD/auxiliary/TypeTraits.hpp:24:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/include/openPMD/auxiliary/UniquePtr.hpp:3:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/functional:515:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__algorithm/search.h:14:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__algorithm/iterator_operations.h:12:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__algorithm/iter_swap.h:14:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__utility/swap.h:16:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/is_nothrow_move_constructible.h:15:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/is_nothrow_constructible.h:17:
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/cstddef:46:5: error: <cstddef> tried including <stddef.h> but didn't find libc++'s <stddef.h> header. This usually means that your header search paths are not configured properly. The header search paths should contain the C++ Standard Library headers before any C Standard Library, and you are probably using compiler flags that make that not be the case.
# error <cstddef> tried including <stddef.h> but didn't find libc++'s <stddef.h> header. \
^
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/src/config.cpp:22:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/include/openPMD/version.hpp:23:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/map:574:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__algorithm/equal.h:14:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__algorithm/unwrap_iter.h:13:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__iterator/iterator_traits.h:14:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__concepts/constructible.h:12:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__concepts/convertible_to.h:13:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/is_convertible.h:14:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/is_array.h:14:
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/cstddef:46:5: error: <cstddef> tried including <stddef.h> but didn't find libc++'s <stddef.h> header. This usually means that your header search paths are not configured properly. The header search paths should contain the C++ Standard Library headers before any C Standard Library, and you are probably using compiler flags that make that not be the case.
# error <cstddef> tried including <stddef.h> but didn't find libc++'s <stddef.h> header. \
^
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/cstddef:59:9: error: no member named 'nullptr_t' in the global namespace
using ::nullptr_t;
~~^
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/cstddef:59:9: error: no member named 'nullptr_t' in the global namespace
using ::nullptr_t;
~~^
In file included from /Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/Base/AMReX_BlockMutex.cpp:1:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/Base/AMReX_BlockMutex.H:5:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/Base/AMReX_Gpu.H:17:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedamrex-src/Src/Base/AMReX_GpuControl.H:8:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/utility:251:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__utility/as_const.h:14:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__utility/forward.h:15:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/remove_reference.h:13:
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/cstddef:46:5: error: <cstddef> tried including <stddef.h> but didn't find libc++'s <stddef.h> header. This usually means that your header search paths are not configured properly. The header search paths should contain the C++ Standard Library headers before any C Standard Library, and you are probably using compiler flags that make that not be the case.
# error <cstddef> tried including <stddef.h> but didn't find libc++'s <stddef.h> header. \
^
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/cstddef:59:9: error: no member named 'nullptr_t' in the global namespace
using ::nullptr_t;
~~^
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/src/config.cpp:22:
In file included from /Users/arun/src/WarpX/build/_deps/fetchedopenpmd-src/include/openPMD/version.hpp:23:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/map:574:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__algorithm/equal.h:14:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__algorithm/unwrap_iter.h:13:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__iterator/iterator_traits.h:14:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__concepts/constructible.h:13:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__concepts/destructible.h:13:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/is_nothrow_destructible.h:16:
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/is_scalar.h:17:
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/is_null_pointer.h:26:28: error: use of undeclared identifier 'nullptr_t'; did you mean 'nullptr'?
struct __is_nullptr_t_impl<nullptr_t> : public true_type {};
^
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/is_null_pointer.h:26:28: error: template argument for template type parameter must be a type
struct __is_nullptr_t_impl<nullptr_t> : public true_type {};
^~~~~~~~~
/Library/Developer/CommandLineTools/SDKs/MacOSX14.4.sdk/usr/include/c++/v1/__type_traits/is_null_pointer.h:23:17: note: template parameter is declared here
template <class _Tp>
I also see errors in regards to <stdint.h>, <stdlib.h>, and <math.h>.
Is this a problem when creating the build scripts or just some issue with my CC-compiler not knowing the path to these files (which seems odd to me)?
Let me know if I should upload any more files or logs.
Thanks for reporting this issue. ~~In the above case, could you let us know how you setup the environment and installed dependencies for WarpX?~~ ~~Did you follow one of the instructions sets here?~~ https://warpx.readthedocs.io/en/latest/install/dependencies.html#install-dependencies
Ah, I see now that you mentioned you used brew. I think that we need to update the installation instructions with brew.
Would you be able to try installing the dependencies with conda or spack?
Thanks for the report @arunpersaud and @RemiLehe for the guidance.
@arunpersaud do you prefer changing WarpX C++ code and compiling that ("developer" install) or do you like to use WarpX and extend it solely with input files and Python workflows ("user" install)? In the latter case, using our pre-compiled conda packages is the fastest way to success: https://warpx.readthedocs.io/en/latest/install/users.html
You currently picked the developer install. From the error, I think your XCode package on macOS needs an update - likely because of a macOS operating system update (e.g., macOS 12 to 13 or 14) since XCode was first installed.
After the update of XCode, it is also likely that your brew needs an update (as in: needs to be removed and re-installed), because as a user-level package manager it cannot "realize" its underlying OS changed.
After the update, remove rm -rf build to reconfigure with CMake.
For macOS and brew, here are a few general macOS hints: https://warpx.readthedocs.io/en/latest/install/users.html#tips-for-macos-users and if you like we could do a quick zoom to check if your less-used brew instructions are still up to date: https://warpx.readthedocs.io/en/latest/install/dependencies.html#brew-macos-linux If you prefer brew, I am happy to get this working in brew with you.
Related issues: We fixed issues on SUSE Linux, which had multiple Python versions installed in the system root path. The best solution is to use a Python venv, e.g.,
python3 -m venv $HOME/venvs/warpx-cpu
source $HOME/venvs/warpx-cpu
and have it active to compile, pip install and run to have exactly one Python narrowed down for usage.
The alternative for a system-wide install with multiple system-wide Python versions is to set a CMake hint:
https://cmake.org/cmake/help/latest/module/FindPython.html#hints
Python_ROOT_DIR only works if the Pythons have a distinct install path, Python_FIND_STRATEGY=VERSION might be a way to go. But I would recommend using a Conda environment or at least venv instead.
We still try to find out what the XCode/Apple Developer SDK/Brew interaction is that is broken on the mac system. Looks like the developer SDK is out of sync (14.4 on a 14.6 system) or so.
Something like https://stackoverflow.com/a/78664080/2719194
Just to confirm that the venv solution worked. One step I had to do for WarpX to compile was pip install wheel inside the venv.
We also got it to run by setting -DPython_EXECUTABLE to the correct version (which on my system had to be the same one that python3 points to, which was python3.11). Setting the exectuable to the path of python3.12 did not work and cmake tried to use python3.11 in some places.
looked a bit more into this and the problem seems to be that
include(${WarpX_SOURCE_DIR}/cmake/dependencies/pybind11.cmake)
looks for python3, which on my computer points to 3.11 and not 3.12 and this doesn't take into account python_EXECUTABLE.
One solution is to add
set(PYBIND11_PYTHON_VERSION 3.12 CACHE STRING "")
before the above include call and setting the python_EXECUTABLE to 3.12
Then limiting from the command line via -DPYBIND11_PYTHON_VERSION=3.12 could work, too.
You are right, Python_EXECUTABLE is not a hint to the FindPython.cmake module anymore, which we use here. I will update the docs.
just confirming that -DPYBIND11_PYTHON_VERSION=3.12 works on my linux box. The full cmake command to configure the build that I used was:
cmake -S . -B build_py -DWarpX_DIMS="1;2;RZ;3" -DWarpX_PYTHON=ON -DPython_EXECUTBALE=/usr/bin/python3.12 -DPYBIND11_PYTHON_VERSION=3.12
Note: Typo -DPython_EXECUTBALE should be -DPython_EXECUTABLE
https://cmake.org/cmake/help/v3.30/module/FindPython.html#artifacts-specification
@arunpersaud I think I found a robust fix for your issue via PR #5390
The regular https://cmake.org/cmake/help/latest/module/FindPython.html logic will now apply and in particular you can set these hints to select a Python prefix path location and these options if multiple are present at the same path/location.
Examples:
cmake -S . -B build_py -DWarpX_DIMS="1;2;RZ;3" -DWarpX_PYTHON=ON -DPython_ROOT_DIR=/usr -DPython_FIND_STRATEGY=LOCATION
or
cmake -S . -B build_py -DWarpX_DIMS="1;2;RZ;3" -DWarpX_PYTHON=ON -DPython_FIND_VERSION=3.12.6 -DPython_FIND_VERSION_EXACT=TRUE
and/or combinations of these.
As before: If you are in a venv or conda environment, the Python present in that one will be taken by default, which is the most robust way to build (because you want to install WarpX/ImpactX as a Python package too and be sure they are well matched to the python you use to run your scripts).