PyMFEM
PyMFEM copied to clipboard
mfem
compilation error using intel compilers
I have been compiling PyMFEM successfully using intel compilers but now I am getting a compilation error. Has anybody faced the same issue?
Five days ago the compilation with intel was still working.
Any clues?
Are you using master branch? It is true that I merged one PR yesterday (or one day before). Can you share the error message you are seeing?
The error happened when using the master branch. After the following command: python setup.py install --with-gslib --with-parallel --ext-only --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiicc
we got the following error:
icc -pthread -B /home/docker/.conda/envs/mfem/compiler_compat -shared -L/home/docker/.conda/envs/mfem/lib -Wl,-z,noexecstack,-z,relro,-z,now,-rpath,$ORIGIN/../..,-rpath,$ORIGIN/../../.. -L/home/docker/.conda/envs/mfem/lib -L/home/docker/.conda/envs/mfem/lib -Wl,-z,noexecstack,-z,relro,-z,now,-rpath,$ORIGIN/../..,-rpath,$ORIGIN/../../.. build/temp.linux-x86_64-3.9/std_vectors_wrap.o -L/home/docker/.conda/envs/mfem/lib/python3.9/site-packages/mfem/external/ser/lib -Wl,-R/home/docker/.conda/envs/mfem/lib/python3.9/site-packages/mfem/external/ser/lib -lmfem -o /home/docker/PyMFEM/mfem/_ser/_std_vectors.cpython-39-x86_64-linux-gnu.so
[91merror: command '/opt/intel/oneapi/compiler/2022.1.0/linux/bin/intel64/icc' failed with exit code 4
It works fine if I specify an older commit for mfem, i.e.,
python setup.py install --with-gslib --with-parallel --ext-only --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiicc --mfem-branch=e7485db5ebc5d40411887aa97563dd6a3722e3d8
I'm also encountering issues when trying to build on a cluster with Intel compilers, but have not yet identified a solution either. My errors are slightly different. Since I already have builds for serial and parallel MFEM, I decided to provide the prefix of the path to the dependencies. When I run
python setup.py install --with-parallel --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpicc --mfemp-prefix="~/Projects/mfem/build_parallel" --hypre-prefix="~/Projects/hypre/src" --metis-prefix="~/Projects/metis-5.1.0" --skip-ext
I get a collection of errors that look like
In file included from pnonlinearform_wrap.cxx(3384): catastrophic error: cannot open source file "_config.hpp" #include "_config.hpp".
The last error that shows up is
File "setup.py", line 1397, in run File "setup.py", line 923, in make_mfem_wrapper File "setup.py", line 286, in make_call File "subprocess.py", line 328, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['python', 'setup.py', 'build_ext', '--inplace', '--parallel', '55']' returned non-zero exit status 1.
So it's a different error message than what you are seeing, but I'm also having problems here.
For more clarity, I am performing the following commands: python setup.py install --with-gslib --with-parallel --ext-only --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiicc python setup.py clean --swig python setup.py install --with-gslib --with-parallel --swig python setup.py install --with-gslib --with-parallel --skip-ext --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiicc
This last step produces the following errors:
icc -pthread -B /home/docker/.conda/envs/mfem/compiler_compat -shared -L/home/docker/.conda/envs/mfem/lib -Wl,-z,noexecstack,-z,relro,-z,now,-rpath,$ORIGIN/../..,-rpath,$ORIGIN/../../.. -L/home/docker/.conda/envs/mfem/lib -L/home/docker/.conda/envs/mfem/lib -Wl,-z,noexecstack,-z,relro,-z,now,-rpath,$ORIGIN/../..,-rpath,$ORIGIN/../../.. build/temp.linux-x86_64-3.9/std_vectors_wrap.o -L/home/docker/.conda/envs/mfem/lib/python3.9/site-packages/mfem/external/ser/lib -Wl,-R/home/docker/.conda/envs/mfem/lib/python3.9/site-packages/mfem/external/ser/lib -lmfem -o /home/docker/PyMFEM/mfem/_ser/_std_vectors.cpython-39-x86_64-linux-gnu.so
[91merror: command '/opt/intel/oneapi/compiler/2022.1.0/linux/bin/intel64/icc' failed with exit code 4
and
nonlininteg_wrap.cxx(3441): catastrophic error: cannot open source file "fem/ceed/interface/operator.hpp" #include "fem/ceed/interface/operator.hpp" ^
compilation aborted for nonlininteg_wrap.cxx (code 4)
I hope this helps.
After many attempts I realized that if you specify the branch=master it works. The commands are as follows: python setup.py install --with-gslib --with-parallel --ext-only --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiicc --mfem-branch=master python setup.py clean --swig python setup.py install --with-gslib --with-parallel --swig python setup.py install --with-gslib --with-parallel --skip-ext --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiicc
Whatever is the default mfem branch in setup.py does not work.
Cheers
@i3s93, @adantra . Thank you for reporting this. The command sequence @adantra regenerate a entire wrapper code, and it is good know that the SWGI recipe files (*.i) are still okay. @i3s93, please let us know if the command sequence works for you, too.
Incidentally, I am working on an update to run SWIG automatically in https://github.com/mfem/PyMFEM/pull/143. I will make sure that it will be tested with Intel compiler before merging.
By the way... When mfem-branch=master is not set. setup.py build MFEM with a specific version of MFEM. Master branch build it using MFEM 4.4 release (specified in https://github.com/mfem/PyMFEM/blob/master/setup.py#L54) Probably I should have modified it to a newer version (see https://github.com/mfem/PyMFEM/blob/autorun_swig_dev/setup.py#L55), when I merged a recent PR.
@sshiraiwa, sorry for the late response. In short, the sequence of commands above works for me in the case of the serial build. I ran your script run_examples.py in serial mode and most of the tests pass, but there seems to be a problem with examples 4 and 25. I believe you are already aware of the latter case. In example 4, it looks like the output is slightly different, but I have not investigated this any further. The parallel build is still not working for me. I have built MFEM and it's dependencies as a separate project, so I already have those library files. What I would like to do is supply the prefix for each of these dependencies, so that the individual pieces do not have to be built every time. I am using the following installation command:
python setup.py install --with-parallel --mfem-source=~/Projects/mfem/ --mfem-prefix=~/Projects/mfem/build_parallel/ --metis-prefix=/opt/apps/intel19/metis/5.1.0/ --hypre-prefix=~/Projects/hypre/src/ --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiicc
When I run the script, the build does look in the right directories and seems to be successful at building some of the wrappers, but then I see this error:
calling ... /home1/apps/intel19/swig/4.0.0/bin/swig -Wall -c++ -python -fastproxy -olddefs -keyword -I/home1/09047/was2489/Projects/mfem/build_parallel/include -I/home1/09047/was2489/Projects/mfem/build_parallel/include/mfem -I/home1/09047/was2489/Projects/mfem -I/home1/09047/was2489/Projects/hypre/src/include -I/opt/apps/intel19/metis/5.1.0/include -I/opt/apps/intel19/impi19_0/python3/3.7.0/lib/python3.7/site-packages/mpi4py/include pfespace.i ../common/hypre_int.i:1: Error: Unable to find 'HYPRE_utilities.h' Failed when calling command: /home1/apps/intel19/swig/4.0.0/bin/swig -Wall -c++ -python -fastproxy -olddefs -keyword -I/home1/09047/was2489/Projects/mfem/build_parallel/include -I/home1/09047/was2489/Projects/mfem/build_parallel/include/mfem -I/home1/09047/was2489/Projects/mfem -I/home1/09047/was2489/Projects/hypre/src/include -I/opt/apps/intel19/metis/5.1.0/include -I/opt/apps/intel19/impi19_0/python3/3.7.0/lib/python3.7/site-packages/mpi4py/include pfespace.i Traceback (most recent call last): File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1614, in <module> main() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1610, in main run_setup() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1598, in run_setup setup( File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup return distutils.core.setup(**attrs) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands self.run_command(cmd) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command cmd_obj.run() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1399, in run _install.run(self) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 72, in run orig.install.run(self) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 692, in run self.run_command('build') File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command self.distribution.run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command cmd_obj.run() File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/command/build.py", line 24, in run super().run() File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run self.run_command(cmd_name) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command self.distribution.run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command cmd_obj.run() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1451, in run generate_wrapper() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 921, in generate_wrapper make_call(command) File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 294, in make_call subprocess.check_call(command, **kwargs) File "/home1/09047/was2489/miniconda3/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/home1/apps/intel19/swig/4.0.0/bin/swig', '-Wall', '-c++', '-python', '-fastproxy', '-olddefs', '-keyword', '-I/home1/09047/was2489/Projects/mfem/build_parallel/include', '-I/home1/09047/was2489/Projects/mfem/build_parallel/include/mfem', '-I/home1/09047/was2489/Projects/mfem', '-I/home1/09047/was2489/Projects/hypre/src/include', '-I/opt/apps/intel19/metis/5.1.0/include', '-I/opt/apps/intel19/impi19_0/python3/3.7.0/lib/python3.7/site-packages/mpi4py/include', 'pfespace.i']' returned non-zero exit status 1.
This error is due to a missing file, but my earlier build for Hypre was successful, so I'm not sure why this file is missing. As I mentioned earlier, I would really like to point the installation script to my other builds for MFEM and its dependencies so that I don't have to keep compiling the source code. Would you happen to have any suggestions for this?
As an alternative to the above, I chose to build MFEM, but point the script to the installations of Hypre and Metis on our cluster:
python setup.py install --with-parallel --metis-prefix=/opt/apps/intel19/metis/5.1.0/ --hypre-prefix=/home1/apps/intel19/impi19_0/petsc/3.14/clx/ --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiicc --mfem-branch=master
This leads to the error message below.
/opt/apps/gcc/8.3.0/bin/ld: cannot find -lGKlib
make[2]: *** [libmfem.so.4.4.0] Error 1
make[2]: Leaving directory `/home1/09047/was2489/Projects/PyMFEM/external/mfem/cmbuild_par'
make[1]: *** [CMakeFiles/mfem.dir/all] Error 2
make[1]: Leaving directory `/home1/09047/was2489/Projects/PyMFEM/external/mfem/cmbuild_par'
make: *** [all] Error 2
Failed when calling command: mfem_parallel
Traceback (most recent call last):
File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1614, in <module>
main()
File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1610, in main
run_setup()
File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1598, in run_setup
setup(
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup
return distutils.core.setup(**attrs)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup
return run_commands(dist)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands
dist.run_commands()
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands
self.run_command(cmd)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
cmd_obj.run()
File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1399, in run
_install.run(self)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 72, in run
orig.install.run(self)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 692, in run
self.run_command('build')
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
self.distribution.run_command(command)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
cmd_obj.run()
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/command/build.py", line 24, in run
super().run()
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run
self.run_command(cmd_name)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command
self.distribution.run_command(command)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command
super().run_command(command)
File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command
cmd_obj.run()
File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1446, in run
cmake_make_mfem(serial=False)
File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 730, in cmake_make_mfem
make('mfem_' + txt)
File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 327, in make
make_call(command, target=target)
File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 294, in make_call
subprocess.check_call(command, **kwargs)
File "/home1/09047/was2489/miniconda3/lib/python3.9/subprocess.py", line 373, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['make', '-j', '55']' returned non-zero exit status 2.
In this case, the script cannot find GKlib, which is apparently a dependency for Metis. I'm not sure why this file is missing because a proper build of Metis would have included this.
I apologize for the long message, but hopefully this gives you a sense of the problems I have encountered.
@i3s93 Thank you. METIS recently changed to put some functionality outside as GKlib. This is caused by an adjustment I introduced due to this change. Actually, handling this change is more complicated than I thought, since depending on the version of METIS, mfem may or may not need to be linked using -GKlib. For now, we are rolling back the setup.py script to specifically use an older version in https://github.com/mfem/tpls. For now, can you please use an older PyMFEM?
@i3s93, @adantra Closing this for now. As mentioned earlier, we rolled back setup.py to point an older version of METIS when building it from setup.py. Please take a look at the current master (https://github.com/mfem/PyMFEM/releases/tag/v4.4.0.1). Note that when compiling with intel the proper option is. --CXX=icpc --MPICXX=mpiicpc. Also, (probably) mpi4py needs to compiled with the same MPI compiler.
Hi Syunichi, its not apparent from this where you have libGK installed. Did you build GK and install in the metis lib directory? It would not have been downloaded and built automatically. It is a new dependency since metic 5. -john
On Thu, September 8, 2022 1:38 am, William Sands wrote:
@sshiraiwa, sorry for the late response. In short, the sequence of commands above works for me in the case of the serial build. I ran your script
run_examples.pyin serial mode and most of the tests pass, but there seems to be a problem with examples 4 and 25. I believe you are already aware of the latter case. In example 4, it looks like the output is slightly different, but I have not investigated this any further. The parallel build is still not working for me. I have built MFEM and it's dependencies as a separate project, so I already have those library files. What I would like to do is supply the prefix for each of these dependencies, so that the individual pieces do not have to be built every time. I am using the following installation command:
python setup.py install --with-parallel --mfem-source=~/Projects/mfem/ --mfem-prefix=~/Projects/mfem/build_parallel/ --metis-prefix=/opt/apps/intel19/metis/5.1.0/ --hypre-prefix=~/Projects/hypre/src/ --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiiccWhen I run the script, the build does look in the right directories and seems to be successful at building some of the wrappers, but then I see this error:
calling ... /home1/apps/intel19/swig/4.0.0/bin/swig -Wall -c++ -python -fastproxy -olddefs -keyword -I/home1/09047/was2489/Projects/mfem/build_parallel/include -I/home1/09047/was2489/Projects/mfem/build_parallel/include/mfem -I/home1/09047/was2489/Projects/mfem -I/home1/09047/was2489/Projects/hypre/src/include -I/opt/apps/intel19/metis/5.1.0/include -I/opt/apps/intel19/impi19_0/python3/3.7.0/lib/python3.7/site-packages/mpi4py/include pfespace.i ../common/hypre_int.i:1: Error: Unable to find 'HYPRE_utilities.h' Failed when calling command: /home1/apps/intel19/swig/4.0.0/bin/swig -Wall -c++ -python -fastproxy -olddefs -keyword -I/home1/09047/was2489/Projects/mfem/build_parallel/include -I/home1/09047/was2489/Projects/mfem/build_parallel/include/mfem -I/home1/09047/was2489/Projects/mfem -I/home1/09047/was2489/Projects/hypre/src/include -I/opt/apps/intel19/metis/5.1.0/include -I/opt/apps/intel19/impi19_0/python3/3.7.0/lib/python3.7/site-packages/mpi4py/include pfespace.i Traceback (most recent call last): File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1614, in <module> main() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1610, in main run_setup() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1598, in run_setup setup( File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup return distutils.core.setup(**attrs) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands self.run_command(cmd) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command cmd_obj.run() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1399, in run _install.run(self) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 72, in run orig.install.run(self) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 692, in run self.run_command('build') File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command self.distribution.run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command cmd_obj.run() File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/command/build.py", line 24, in run super().run() File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run self.run_command(cmd_name) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command self.distribution.run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command cmd_obj.run() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1451, in run generate_wrapper() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 921, in generate_wrapper make_call(command) File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 294, in make_call subprocess.check_call(command, **kwargs) File "/home1/09047/was2489/miniconda3/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['/home1/apps/intel19/swig/4.0.0/bin/swig', '-Wall', '-c++', '-python', '-fastproxy', '-olddefs', '-keyword', '-I/home1/09047/was2489/Projects/mfem/build_parallel/include', '-I/home1/09047/was2489/Projects/mfem/build_parallel/include/mfem', '-I/home1/09047/was2489/Projects/mfem', '-I/home1/09047/was2489/Projects/hypre/src/include', '-I/opt/apps/intel19/metis/5.1.0/include', '-I/opt/apps/intel19/impi19_0/python3/3.7.0/lib/python3.7/site-packages/mpi4py/include', 'pfespace.i']' returned non-zero exit status 1.This error is due to a missing file, but my earlier build for Hypre was successful, so I'm not sure why this file is missing. As I mentioned earlier, I would really like to point the installation script to my other builds for MFEM and its dependencies so that I don't have to keep compiling the source code. Would you happen to have any suggestions for this?
As an alternative to the above, I chose to build MFEM, but point the script to the installations of Hypre and Metis on our cluster:
python setup.py install --with-parallel --metis-prefix=/opt/apps/intel19/metis/5.1.0/ --hypre-prefix=/home1/apps/intel19/impi19_0/petsc/3.14/clx/ --CC=icc --CXX=icc --MPICC=mpiicc --MPICXX=mpiicc --mfem-branch=masterThis leads to the error message below.
/opt/apps/gcc/8.3.0/bin/ld: cannot find -lGKlib make[2]: *** [libmfem.so.4.4.0] Error 1 make[2]: Leaving directory `/home1/09047/was2489/Projects/PyMFEM/external/mfem/cmbuild_par' make[1]: *** [CMakeFiles/mfem.dir/all] Error 2 make[1]: Leaving directory `/home1/09047/was2489/Projects/PyMFEM/external/mfem/cmbuild_par' make: *** [all] Error 2 Failed when calling command: mfem_parallel Traceback (most recent call last): File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1614, in <module> main() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1610, in main run_setup() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1598, in run_setup setup( File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/__init__.py", line 87, in setup return distutils.core.setup(**attrs) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 185, in setup return run_commands(dist) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/core.py", line 201, in run_commands dist.run_commands() File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 973, in run_commands self.run_command(cmd) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command cmd_obj.run() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1399, in run _install.run(self) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/command/install.py", line 72, in run orig.install.run(self) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/command/install.py", line 692, in run self.run_command('build') File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command self.distribution.run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command cmd_obj.run() File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/command/build.py", line 24, in run super().run() File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/command/build.py", line 132, in run self.run_command(cmd_name) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/cmd.py", line 319, in run_command self.distribution.run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/dist.py", line 1217, in run_command super().run_command(command) File "/home1/09047/was2489/miniconda3/lib/python3.9/site-packages/setuptools/_distutils/dist.py", line 992, in run_command cmd_obj.run() File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 1446, in run cmake_make_mfem(serial=False) File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 730, in cmake_make_mfem make('mfem_' + txt) File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 327, in make make_call(command, target=target) File "/home1/09047/was2489/Projects/PyMFEM/setup.py", line 294, in make_call subprocess.check_call(command, **kwargs) File "/home1/09047/was2489/miniconda3/lib/python3.9/subprocess.py", line 373, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['make', '-j', '55']' returned non-zero exit status 2.In this case, the script cannot find
GKlib, which is apparently dependency for Metis. I'm not sure why this file is missing because a proper build of Metis would have included this dependency.I apologize for the long message, but hopefully this gives you a sense of the problems I have encountered.
-- Reply to this email directly or view it on GitHub: https://github.com/mfem/PyMFEM/issues/144#issuecomment-1240243944 You are receiving this because you are subscribed to this thread.
Message ID: @.***>
-- Principal Research Scientist John Wright Office 617-253-9612 zoom: https://mit.zoom.us/my/jcwright
Hi, @jcwright77. We are using .tar.gz file of metis under https://github.com/mfem/tpls, This includes libGK. This is an older form of how METIS5 was packaged. It is not necessary to download libGK separately.
As a side note, in setup.py, if you change the following flag to True
use_metis_gklib = False
then, it downloads both libGK and METIS from Github repo. It also makes some adjustment to a link flag when building MFEM so that libmfem.so is linked both libGK and METIS. However, at moment we are not testing this feature well, and it is not recommended to use it.