JohannesBuchner
Unable to run PyMultiNest due to ValueError
Hello. I am trying to run a script (hotokezaka_bns_inference_test.py) on my Mac and am currently unable to do so; I've been stuck on this error for about a week, and am wondering if you can provide any insight:
Traceback (most recent call last):
File "/Users/meem/Desktop/kn_pe_em/gemma-dev-main/hotokezaka_bns_inference_test.py", line 1, in numpy.ufunc."
ValueError: All ufuncs must have type numpy.ufunc. Received (<ufunc 'sph_legendre_p'>, <ufunc 'sph_legendre_p'>, <ufunc 'sph_legendre_p'>)
If it's relevant, I was also receiving this message previously, but now I've 'graduated' to only receiving the above error:
Traceback (most recent call last):
File "/Users/meem/Desktop/kn_pe_em/gemma-dev-main/Amelia.py", line 109, in
I've never opened an issue like this before, so I'm not sure what additional information is helpful to provide --- just let me know. From my debugging, I've gathered that this is likely either an issue with conflicting versions of numpy and scipy on my computer, or something to do with not having a fortran compiler?
Confusingly, PyMultiNest is working elsewhere on my computer, in another version of a repo I have (entitled GEMMA).
Try creating a fresh conda environment, and install pymultinest, then test it with one of the demo scripts.
Hello! Thank you for your comment; I tried that and have been unable to complete the (Py)MultiNest installation on my computer, due to an error when I run cmake .. in MultiNest/build (am following the installation instructions here:
(base) meem@U-C02G53NGQ05N build % cmake ..
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.10 will be removed from a future version of
CMake.
Update the VERSION argument <min> value. Or, use the <min>...<max> syntax
to tell CMake that the project requires at least <min> but has been updated
to work with policies introduced by <max> or earlier.
-- The Fortran compiler identification is GNU 14.2.0
-- The C compiler identification is AppleClang 17.0.0.17000013
-- The CXX compiler identification is AppleClang 17.0.0.17000013
-- Checking whether Fortran compiler has -isysroot
-- Checking whether Fortran compiler has -isysroot - yes
-- Checking whether Fortran compiler supports OSX deployment target flag
-- Checking whether Fortran compiler supports OSX deployment target flag - yes
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /opt/homebrew/bin/gfortran - skipped
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at src/CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 3.5 has been removed from CMake.
Update the VERSION argument <min> value. Or, use the <min>...<max> syntax
to tell CMake that the project requires at least <min> but has been updated
to work with policies introduced by <max> or earlier.
Or, add -DCMAKE_POLICY_VERSION_MINIMUM=3.5 to try configuring anyway.
-- Configuring incomplete, errors occurred!
I'm wondering if you can provide any advice or have seen this before, please?
If it is helpful, this is what is inside CMakeLists.txt:
cmake_minimum_required(VERSION 3.5)
project(multinest Fortran C CXX)
# By default CMake will install MultiNest to /usr/local on Linux/Mac
# unless you override the install location manually. You can do this
# either by either running:
# cmake -DCMAKE_INSTALL_PREFIX=/path/to/dir ..
# or by uncommenting and modifying this variable:
#SET(CMAKE_INSTALL_PREFIX /usr/local/multinest)
#=============================================================================
# All done editing
SET(MultiNest_VERSION "3.10")
SET(EXECUTABLE_OUTPUT_PATH ${CMAKE_SOURCE_DIR}/bin)
SET(LIBRARY_OUTPUT_PATH ${CMAKE_SOURCE_DIR}/lib)
include_directories(${CMAKE_SOURCE_DIR}/include)
add_subdirectory(src)
I guess it wants you to change cmake_minimum_required(VERSION 3.5) to cmake_minimum_required(VERSION 3.5...3.10)?
Now I receive the following error message:
(base) meem@U-C02G53NGQ05N build % cmake ..
-- Detected gfortran, adding -ffree-line-length-none compiler flag.
-- Detected gfortran >= 10, adding -std=legacy compiler flag.
-- Detected gfortran-10+, adding -w -fallow-argument-mismatch compiler flag.
-- Could NOT find MPI_C (missing: MPI_C_WORKS)
-- Could NOT find MPI_CXX (missing: MPI_CXX_WORKS)
-- Could NOT find MPI_Fortran (missing: MPI_Fortran_WORKS)
-- Could NOT find MPI (missing: MPI_C_FOUND MPI_CXX_FOUND MPI_Fortran_FOUND)
-- MPI not found, only non-MPI MultiNest libraries will be built.
CMake Error at src/example_eggbox_C/CMakeLists.txt:7 (FIND_LIBRARY):
Could not find m using the following names:
-- Configuring incomplete, errors occurred!
Hello,
As an update, I was able to complete installing MultiNest and PyMultiNest according to the installation instructions by manually adding the suggested fix to every CMakeLists.txt file in MultiNest (including all those in the directories in /src/).
However, when I try to run PyMultiNest on my computer, I receive the following error message. I'm curious if you can advise? This script used to run without errors in the past.
(base) meem@U-C02G53NGQ05N lc_sampling % python bilby_m1_m2_r.py
/Users/meem/opt/anaconda3/lib/python3.8/site-packages/astropy/units/quantity.py:673: RuntimeWarning: overflow encountered in exp
result = super().__array_ufunc__(function, method, *arrays, **kwargs)
11:46 bilby INFO : Running for label 'INkd_Sdk_14_14_10p0_May_07_25NewM1Prior', output will be saved to 'bilby-pymultinest-m1m2r'
problem: dlopen(/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib, 0x0006): tried: '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/libmultinest.dylib' (no such file), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib' (no such file), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.3.10.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/libmultinest.3.10.dylib' (no such file), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.3.10.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.3.10.dylib' (no such file), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.3.10.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))
Traceback (most recent call last):
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/pymultinest/run.py", line 16, in _load_library
return cdll.LoadLibrary(libname)
File "/Users/meem/opt/anaconda3/lib/python3.8/ctypes/__init__.py", line 459, in LoadLibrary
return self._dlltype(name)
File "/Users/meem/opt/anaconda3/lib/python3.8/ctypes/__init__.py", line 381, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib, 0x0006): tried: '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/libmultinest.dylib' (no such file), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib' (no such file), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.3.10.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/libmultinest.3.10.dylib' (no such file), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.3.10.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/System/Volumes/Preboot/Cryptexes/OS/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.3.10.dylib' (no such file), '/Users/meem/opt/anaconda3/lib/python3.8/site-packages/MultiNest/lib/libmultinest.3.10.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/bilby/core/sampler/base_sampler.py", line 319, in _verify_external_sampler
__import__(external_sampler_name)
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/pymultinest/__init__.py", line 19, in <module>
from .run import run
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/pymultinest/run.py", line 52, in <module>
lib = _load_library('libmultinest')
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/pymultinest/run.py", line 50, in _load_library
sys.exit(1)
SystemExit: 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "bilby_m1_m2_r.py", line 197, in <module>
result = bilby.run_sampler(
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/bilby/core/sampler/__init__.py", line 190, in run_sampler
sampler = sampler_class(
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/bilby/core/sampler/pymultinest.py", line 79, in __init__
super(Pymultinest, self).__init__(
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/bilby/core/sampler/base_sampler.py", line 904, in __init__
super(_TemporaryFileSamplerMixin, self).__init__(**kwargs)
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/bilby/core/sampler/base_sampler.py", line 238, in __init__
self._verify_external_sampler()
File "/Users/meem/opt/anaconda3/lib/python3.8/site-packages/bilby/core/sampler/base_sampler.py", line 321, in _verify_external_sampler
raise SamplerNotInstalledError(
bilby.core.sampler.base_sampler.SamplerNotInstalledError: Sampler pymultinest is not installed on this system
Looks like the compiler used a different (python) environment than when you are running it?
@ameliahenkel Can you try using these commands,
If you are on MacOS make sure to have gfortran and gcc and export their paths
FC=$(which gfortran-14)
CC=$(which gcc)
CXX=$(which g++)
you may have to change 14 to your gfortran version or maybe remove it completely based on your system.
And if you are on Linux directly do that following
git clone https://github.com/JohannesBuchner/MultiNest && cd MultiNest/build && rm -rf * && cmake -DCMAKE_POLICY_VERSION_MINIMUM=3.5 .. && make && cd ../..
LD_LIBRARY_PATH=$YOURDIR/MultiNest/lib:$LD_LIBRARY_PATH
DYLD_LIBRARY_PATH=$YOURDIR/MultiNest/lib:$DYLD_LIBRARY_PATH
And finally
pip install pymultinest
On Mac Silicon, it appears that pip install pymultinest did not properly install the package. Using conda install -c conda-forge pymultinest solved all my issues. I put together this installation guide, in case it is helpful for anyone else.
Instructions for DFENS Installation on Mac Silicon, with backends homebrew, conda, pip brew install cmake brew install gcc brew install open-mpi
conda env create -f xyz.yml # contains pip install mpi4py and other standard packages cd Documents/GitHub # This is locally where I wanted MultiNest to reside git clone https://github.com/JohannesBuchner/MultiNest cd MultiNest/build cmake ..cmake -DCMAKE_POLICY_VERSION_MINIMUM=3.5 .. # New step required given deprecation, pure cmake does not work cp -v ~/Documents/GitHub/MultiNest/lib/lib* ~/anaconda3/lib/ # New step required given deprecation conda activate xyz conda install -c conda-forge pymultinest # THIS IS IMPORTANT: IT MUST BE CONDA-FORGE INSTALLED, NOT PIP python pymultinest_demo.py # Download this https://github.com/JohannesBuchner/PyMultiNest/blob/master/pymultinest_demo.py and check that it runs properly Finally, add this to your .bashrc profile on Mac. export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/Users/yourname/GitHub/MultiNest/lib export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:/Users/yourname/GitHub/MultiNest/lib
