mfem
MFEM compiles with no errors standalone but pip install mfem crashes
Good evening everyone and @sshiraiwa !
I'm opening this issue because I can compile MFEM without any issue, but I've had a really hard time with pymfem recently.
My config is:
- mac OS Monterey, version 12.4,
- llvm v15.0.6 with clang & clang++ (target: arm64-apple-darwin21.5.0),
- open-mpi v4.1.4_2,
- python 3.10.9,
- swig 4.1.0.
With this config, MFEM (pointing to branch master, d84884d13) installs without any issue as the following log proves :
-- The CXX compiler identification is Clang 15.0.6
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/homebrew/opt/llvm/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found MPI_CXX: /opt/homebrew/Cellar/open-mpi/4.1.4_2/lib/libmpi.dylib (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- Looking for HYPRE ...
-- in HYPRE_DIR = xxx/thirdparty/hypre/src/hypre
-- Performing Test HYPRE_USING_HIP
-- Performing Test HYPRE_USING_HIP - Failed
-- Found HYPRE: xxx/thirdparty/hypre/src/hypre/lib/libHYPRE.dylib
-- HYPRE_INCLUDE_DIRS=xxx/thirdparty/hypre/src/hypre/include
-- Found HYPRE version 22600
-- Found OpenMP_CXX: -Xclang -fopenmp (found version "5.0")
-- Found OpenMP: TRUE (found version "5.0")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- MFEM: using package OPENMP
-- MFEM: using package HYPRE
-- MFEM: using package METIS
-- MFEM: using package MPI_CXX
-- MFEM: using package OpenMP
-- MFEM build type: CMAKE_BUILD_TYPE = Release
-- MFEM version: v4.5.1
-- MFEM git string: remotes/origin/HEAD-0-gd84884d13c20bd54b12683c1f19abaaa7e33877b
-- TPL_INCLUDE_DIRS = /opt/homebrew/opt/libomp/include;xxx/thirdparty/hypre/src/hypre/include;xxx/dependencies_install_scripts/../thirdparty/metis-5.1.0/build/include;/opt/homebrew/Cellar/open-mpi/4.1.4_2/include
-- Writing substitute header --> "mfem.hpp"
-- Writing substitute header --> "mfem-performance.hpp"
-- performance_ex1: add flags "-mcpu=apple-m1 -fcolor-diagnostics -fvectorize -fslp-vectorize -ffp-contract=fast"
-- performance_ex1p: add flags "-mcpu=apple-m1 -fcolor-diagnostics -fvectorize -fslp-vectorize -ffp-contract=fast"
-- Found Doxygen: /opt/homebrew/bin/doxygen (found version "1.9.5") found components: doxygen dot
-- CMAKE_INSTALL_PREFIX = /usr/local
-- Configuring done
-- Generating done
CMake Warning:
Manually-specified variables were not used by the project:
OpenMP_C_FLAGS
OpenMP_C_INCLUDE_DIR
OpenMP_C_LIB_NAMES
-- Build files have been written to: xxx/thirdparty/mfem/build
[ 3%] Building CXX object CMakeFiles/mfem.dir/general/array.cpp.o
[ 3%] Building CXX object CMakeFiles/mfem.dir/general/device.cpp.o
[ 3%] Building CXX object CMakeFiles/mfem.dir/general/error.cpp.o
...
[ 96%] Built target mfem
[ 96%] Building CXX object examples/CMakeFiles/ex1p.dir/ex1p.cpp.o
[100%] Linking CXX executable ex1p
[100%] Built target ex1p
Test project xxx/thirdparty/mfem/build
Start 46: ex1p_np=2
1/1 Test #46: ex1p_np=2 ........................ Passed 0.69 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 0.70 sec
However, I cannot, whatever I try, install PyMFEM ... 😭😭😭 I tried:
- pip install mfem==4.4.0.3 --> fails because of SWIG,
- ~~pip install mfem==4.4.0.4 --install-option="--mfem-branch=master" --> fails because of obvious incompatibilities between pymfem 4.4 & mfem 4.5, nevermind ~~
- cloning the repo and pointing to mfem_45_dev
- pip install .,
- pip install . --install-option="--mfem-branch=master", both fail because of a bunch of errors like this one:
/opt/homebrew/opt/llvm/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX12.sdk -DTARGET_PY3=1 -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/mfem/external/ser/include -I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/mfem/external/ser/include/mfem -I/Users/tbridel/Documents/1-CODES/17-PyMFEM/external/mfem -I/opt/homebrew/lib/python3.10/site-packages/numpy/core/include -I/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.10/include/python3.10 -c submesh_wrap.cxx -o build/temp.macosx-12-arm64-cpython-310/submesh_wrap.o -std=c++11 -DSWIG_TYPE_TABLE=PyMFEM
submesh_wrap.cxx:3831:39: error: call to implicitly-deleted copy constructor of 'mfem::TransferMap'
resultobj = SWIG_NewPointerObj((new mfem::TransferMap(static_cast< const mfem::TransferMap& >(result))), SWIGTYPE_p_mfem__TransferMap, SWIG_POINTER_OWN | 0 );
^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
submesh_wrap.cxx:1093:89: note: expanded from macro 'SWIG_NewPointerObj'
#define SWIG_NewPointerObj(ptr, type, flags) SWIG_Python_NewPointerObj(NULL, ptr, type, flags)
^~~
/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/mfem/external/ser/include/mfem/mesh/submesh/transfermap.hpp:69:46: note: copy constructor of 'TransferMap' is implicitly deleted because field 'root_fes_' has a deleted copy constructor
std::unique_ptr<const FiniteElementSpace> root_fes_;
^
/opt/homebrew/opt/llvm/bin/../include/c++/v1/__memory/unique_ptr.h:201:3: note: copy constructor is implicitly deleted because 'unique_ptr<const mfem::FiniteElementSpace>' has a user-declared move constructor
unique_ptr(unique_ptr&& __u) _NOEXCEPT
^
submesh_wrap.cxx:33:69: error: call to implicitly-deleted copy constructor of 'mfem::TransferMap'
SwigValueWrapper& operator=(const T& t) { SwigMovePointer tmp(new T(t)); pointer = tmp; return *this; }
^ ~
submesh_wrap.cxx:3814:14: note: in instantiation of member function 'SwigValueWrapper<mfem::TransferMap>::operator=' requested here
result = mfem::SubMesh::CreateTransferMap((mfem::GridFunction const &)*arg1,(mfem::GridFunction const &)*arg2);
^
/opt/homebrew/opt/[email protected]/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/mfem/external/ser/include/mfem/mesh/submesh/transfermap.hpp:69:46: note: copy constructor of 'TransferMap' is implicitly deleted because field 'root_fes_' has a deleted copy constructor
std::unique_ptr<const FiniteElementSpace> root_fes_;
^
/opt/homebrew/opt/llvm/bin/../include/c++/v1/__memory/unique_ptr.h:201:3: note: copy constructor is implicitly deleted because 'unique_ptr<const mfem::FiniteElementSpace>' has a user-declared move constructor
unique_ptr(unique_ptr&& __u) _NOEXCEPT
^
2 errors generated.
These start happening during the calling ... /opt/homebrew/opt/[email protected]/bin/python3.10 setup.py build_ext --inplace step of the building process - see attached log for more info.
pymfem.log
Do you have any idea what could be going on here ? I feel like, MFEM v4.5 installing without any issues at all, the python wrapper should also get there, shouldn't it ?
Thank you very much for your help !!
Hi @tbridel. Thank you for reporting this error. TransferMap is a new feature in MFEM 4.5, and a new version of wrapper is still in-progress.
What is happening is that since CreateTransferMap returns an object as value, not as a pointer, SWIG is trying to so something smart, which is SwigValueWrapper. it seems it is relating to a move constructor, which I thought @pazner worked out a few days ago (https://github.com/mfem/mfem/pull/3402). I was hoping this change was sufficient to avoid an error. But, there may be something else going on Mac.
Strange, it works for me, running:
- macOS 12.6.1
- Python 3.10.8
- SWIG 4.1.1
In particular, the problematic line is
new mfem::TransferMap(static_cast< const mfem::TransferMap& >(result))
but in the version of submesh_wrap.cxx that is generated for me, I have instead
new mfem::TransferMap(result)
(the static_cast to a reference type is preventing the use of the move constructor, hence the error about the copy constructor).
Any idea why there would be a difference in the generated code? (I tried with SWIG 4.1.0 and it works for me too)
Hi @pazner and @sshiraiwa !
Thank you for your answers.
@pazner Are you using llvm compilers or GNU ? Because indeed I only see that possible difference 🤔 And also the fact that you updated your Mac OS which I haven't done yet 🤔
I tried with the latest gcc and I have the same problem, also I see the same generated code:
submesh_wrap.cxx:3831:39: error: call to implicitly-deleted copy constructor of 'mfem::TransferMap'
resultobj = SWIG_NewPointerObj((new mfem::TransferMap(static_cast< const mfem::TransferMap& >(result))), SWIGTYPE_p_mfem__TransferMap, SWIG_POINTER_OWN | 0 );
Ah ! @pazner my version of the SWIG python wrapper was not aligned with my SWIG install -- I had swig v4.1.1 but the python wrapper was at 4.0.2. This made it possible for me to install the sequential version 🆗 . EDIT: and also made it possible to install the parallel version 🆗 .
Ok so on branch mfem_45_dev and asking for --mfem-branch=master, with the latest LLVM & python 3.10.9, I can do it with the exception of the small patch in setup.py regarding SED (which is messed up on mac).
Hi @tbridel I am glad things are okay now. SWIG had an issue with move constructor and apparently it was fixed rather recently (https://github.com/swig/swig/issues/1044). This also means that we need to update SWIG version requirement !
Thank you for the patch. If possible, can you also make PR about not using sed, so that we can have your contribution appears in git?
Hello @sshiraiwa
Sure, give me a moment, I'll post a PR as soon as time permits !