TASMANIAN
TASMANIAN copied to clipboard
ORNL
Build Tasmanian for apple m1
Hi,
First and foremost, I'd want to express my gratitude to the developer for their excellent job. I've utilized the Tasmanian library for several of my research projects. I recently acquired a machine with an Apple m1 processor. Installing Tasmanian straight from pip results in a broken library. I attempted to build the package using the build instructions. While the package was successfully built, I had the following issue while attempting to import Tasmanian:
OSError Traceback (most recent call last)
<ipython-input-2-405aa3b9f3e0> in <module>
----> 1 import Tasmanian
/usr/local/share/Tasmanian/python/Tasmanian.py in <module>
43 import TasmanianDREAM as DREAM
44
---> 45 from TasmanianAddons import *
46
47 def useVerboseErrors(bVerbose):
/usr/local/share/Tasmanian/python/TasmanianAddons.py in <module>
37 TasmanianInputError = TasmanianConfig.TasmanianInputError
38
---> 39 pLibCTSG = cdll.LoadLibrary(TasmanianConfig.__path_libcaddons__)
40
41 type_1Dfunc = CFUNCTYPE(c_double, c_double)
/opt/homebrew/Caskroom/miniforge/base/envs/automatic_projection_filter/lib/python3.9/ctypes/__init__.py in LoadLibrary(self, name)
458
459 def LoadLibrary(self, name):
--> 460 return self._dlltype(name)
461
462 __class_getitem__ = classmethod(_types.GenericAlias)
/opt/homebrew/Caskroom/miniforge/base/envs/automatic_projection_filter/lib/python3.9/ctypes/__init__.py in __init__(self, name, mode, handle, use_errno, use_last_error, winmode)
380
381 if handle is None:
--> 382 self._handle = _dlopen(self._name, mode)
383 else:
384 self._handle = handle
OSError: dlopen(/usr/local/lib/libtasmaniancaddons.dylib, 0x0006): weak-def symbol not found '__ZN7TasGrid20TasmanianDenseSolver18solvesLeastSquaresINSt3__17complexIdEEEEvPKNS_19AccelerationContextEiiPT_iS9_'
How to solve this problem? Thanks
Hey, I'm glad you find Tasmanian useful.
The build is missing some symbol. I recommend against installing in /usr/local since it's hard to uninstall, but it should still work.
- Are you using the latest master or the latest release?
- What happens if you run:
/usr/local/bin/tasgrid -v
- When you build it with CMake, did you use BLAS or ENABL_RECOMMENDED? You can try to skip ENABLE_RECOMMENDED and try to manually enable or disable BLAS.
- Do you know if M1 used clang or gcc?
Hi,
Thank you for your response. I am not an expert in compiling C/C++ programs from source code. As requested, here is some additional information:
- I am using the master branch from Github.
- Here is the output from `/usr/local/bin/tasgrid -v':
Tasmanian Sparse Grids version: 7.8 (development)
git commit hash: 1a7e80ebb387a225cc8c6d311f9a26589a4fc942
cmake cxx flags: Debug,
license: BSD 3-Clause with UT-Battelle disclaimer
OpenMP multithreading: Disabled
GPU backend framework: none
Available acceleration: none
- I use the following cmake configuration
cmake -D Tasmanian_ENABLE_RECOMMENDED:BOOL=ON ..
-- Checking for LAPACK LQ factorization: not found (using QR instead)
--
-- Tasmanian 7.8 (development): summary of build options
-- -D CMAKE_BUILD_TYPE:STRING=Debug
-- -D CMAKE_INSTALL_PREFIX:PATH=/usr/local
-- -D CMAKE_CXX_FLAGS:STRING=
-- -D BUILD_SHARED_LIBS=ON
-- -D Tasmanian_ENABLE_OPENMP:BOOL=TRUE
-- -D Tasmanian_ENABLE_BLAS:BOOL=ON
-- -D Tasmanian_ENABLE_MPI:BOOL=OFF
-- -D Tasmanian_ENABLE_PYTHON:BOOL=ON
-- -D Tasmanian_ENABLE_CUDA:BOOL=OFF
-- -D Tasmanian_ENABLE_MAGMA:BOOL=OFF
-- -D Tasmanian_ENABLE_HIP:BOOL=OFF
-- -D Tasmanian_ENABLE_DPCPP:BOOL=OFF
-- -D Tasmanian_ENABLE_SWIG:BOOL=OFF
-- -D Tasmanian_ENABLE_FORTRAN:BOOL=OFF
-- -D Tasmanian_ENABLE_DOXYGEN:BOOL=OFF
--
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/foo/GitHub/TAS
- I'm not sure if clang or gcc is utilized for compilation. How can this be verified?
❯ clang++ --version
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: arm64-apple-darwin21.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
❯ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 13.0.0 (clang-1300.0.29.3)
Target: arm64-apple-darwin21.1.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
Thanks.
That's very strange, CMake says that BLAS has been enabled, but tasgrid doesn't show cpu_blas as an available acceleration mode.
Delete everything in your build folder and try using the following CMake command:
cmake -DCMAKE_BUILD_TYPE=Release -DTasmanian_ENABLE_PYTHON=ON ..
That will explicitly disable BLAS and hopefully things will work fine.
I have seen other BLAS issues on Apple even on the Intel chips. I may have to look into this.
I tried your advice. Regrettably, it does not resolve the issue. 'make test' failed again on the python part:
The following tests FAILED:
14 - PythonIO (Failed)
15 - PythonAcceleration (Failed)
16 - PythonExceptions (Failed)
17 - PythonMakeUpdate (Failed)
18 - PythonRefine (Failed)
19 - PythonLearning (Failed)
20 - PythonMisc (Failed)
21 - PythonAddons (Failed)
22 - PythonDream (Failed)
Checking the log file, the same issue still persists:
22/22 Testing: PythonDream
22/22 Test: PythonDream
Command: "/opt/homebrew/Caskroom/miniforge/base/bin/python" "/Users/foo/GitHub/TASMANIAN/Build/Python/testTSG.py" "TestTasmanian.testDream"
Directory: /Users/foo/GitHub/TASMANIAN/Build/Python
"PythonDream" start time: Nov 27 19:26 +03
Output:
----------------------------------------------------------
Traceback (most recent call last):
File "/Users/foo/GitHub/TASMANIAN/Build/Python/testTSG.py", line 17, in <module>
import testExceptions
File "/Users/foo/GitHub/TASMANIAN/Build/Python/testExceptions.py", line 2, in <module>
import Tasmanian
File "/Users/foo/GitHub/TASMANIAN/Build/Python/Tasmanian.py", line 45, in <module>
from TasmanianAddons import *
File "/Users/foo/GitHub/TASMANIAN/Build/Python/TasmanianAddons.py", line 39, in <module>
pLibCTSG = cdll.LoadLibrary(TasmanianConfig.__path_libcaddons__)
File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/ctypes/__init__.py", line 460, in LoadLibrary
return self._dlltype(name)
File "/opt/homebrew/Caskroom/miniforge/base/lib/python3.9/ctypes/__init__.py", line 382, in __init__
self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/foo/GitHub/TASMANIAN/Build/Python/../Addons/libtasmaniancaddons.dylib, 0x0006): weak-def symbol not found '__ZN7TasGrid20TasmanianDenseSolver18solvesLeastSquaresINSt3__17complexIdEEEEvPKNS_19AccelerationContextEiiPT_iS9_'
<end of output>
Test time = 0.19 sec
----------------------------------------------------------
Test Failed.
"PythonDream" end time: Nov 27 19:26 +03
"PythonDream" time elapsed: 00:00:00
----------------------------------------------------------
Thank you.
The short version of the error is that one Tasmanian library could not find something that it needs from another Tasmanian library and/or symbol.
Not sure if this is due to some issue with OSX 11, M1, CMake, or all three.
I'll try to explore this further, but in the mean time you can try to manually set the paths to search for the libraries:
export TasmanianLibs=/Users/foo/GitHub/TASMANIAN/Build/Python/../Addons/:/Users/foo/GitHub/TASMANIAN/Build/Python/../DREAM/:/Users/foo/GitHub/TASMANIAN/Build/Python/../SparseGrids/
export LIBRARY_PATH=$LIBRARY_PATH:$TasmanianLibs
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$TasmanianLibs
export DYLD_LIBRARY_PATH=$DYLD_LIBRARY_PATH:$TasmanianLibs
make test
You can also try to ignore "make test" and go straight for:
make install
make test_install
If the test-install works fine, then Tasmanian should be fine to use.
Hi,
Thank you for your advice. That, too, did not work. I spend some investigation to determine how to list weak symbols from a dylib file. This is what I get:
❯ nm -a libtasmaniancaddons.7.8.0.dylib | grep __ZN7TasGrid20TasmanianDenseSolver18solvesLeastSquaresINSt3__17complexIdEEEEvPKNS_19AccelerationContextEiiPT_iS9_
U __ZN7TasGrid20TasmanianDenseSolver18solvesLeastSquaresINSt3__17complexIdEEEEvPKNS_19AccelerationContextEiiPT_iS9_
which means that the symbol is undefined in the libtasmaniancaddons.7.8.0.dylib file. I'm guessing that has something to do with compiler settings?
The TasmanianDenseSolver::solvesLeastSquares method is a template that is declared in tsgLinearSolvers.hpp then implemented and explicitly instantiated in tsgLinearSolvers.cpp which is then compiled inside libtasmaniansparsegrid.dylib. Either the library isn't found or the symbol is missing from the sparse grid library for some unknown reason, it may even be that the compiler is picking a wrong version of Tasmanian installed somewhere else in your system. It's really hard to debug this without an M1 for me to use.
Can you please try this with 7.7 which is the latest stable release as opposed to 7.8 which is the development version.
Hi, I tested the method using version 7.7. Unfortunately, the same issue still appears. Hopefully, you'll be able to test TASMANIAN for the M1 machine in the future. Thank you.
For what it's worth, I'm encountering the same issue (down to the weak-def symbol) on an Intel-based Mac running Monterey. The issue persists after manually installing a GNU compiler other than AppleClang. I did not have this issue on Catalina with an Intel chip.
That should make it easier to debug, Intel Macs are easier to find.
We have OSX 11 in the CI, but not OSX 12. I'll look into it.
Finally found the problem:
-
The issue was always specific to Python, C++ and the other interfaces were never affected.
-
Unlike Windows, Linux and Intel OSX systems, Python on apple will not make symbols from loaded libraries available globally. The solution is to explicitly load the libraries with the command:
pLibTSG = ctypes.CDLL(__path_libsparsegrid__, mode = ctypes.RTLD_GLOBAL)
- As if the problem isn't enough, there is a second issue with the way arguments are defined for the C bindings. If there is a mismatch between the declared list of arguments and the ones actually used in the call, Python will not issue a proper warning or error message (only any operating system). Instead, Linux/Windows and Intel OSX will ignore the declared arguments and make the call with the provides list. If the list matches whatever the function expects, everything will work fine. However, in the case of a mismatch, the Apple silicon will simply segfault ...
It will be a few days before I update main repo and a few more before I update the pip-installer, but now that I have access to apple silicon, things will work.
Thanks to all that reported the issue and I'm sorry it took this long.