plyvel
plyvel copied to clipboard
wbolster
installation failed on mac osx 10.15
Hello, I know that there are certain issues with the Mac OSX and their version of the GCC compiler. I did install the Gnu GCC, but all the recommended methods of convincing OSX to use the Gnu compiler have failed.
I tried to install plyvel with pip and got fail messages. This is what I get - at least the essential bit.
Complete output (16 lines): running install running build running build_py creating build creating build/lib.macosx-10.6-intel-3.7 creating build/lib.macosx-10.6-intel-3.7/plyvel copying plyvel/_version.py -> build/lib.macosx-10.6-intel-3.7/plyvel copying plyvel/__init__.py -> build/lib.macosx-10.6-intel-3.7/plyvel running build_ext building 'plyvel._plyvel' extension creating build/temp.macosx-10.6-intel-3.7 creating build/temp.macosx-10.6-intel-3.7/plyvel gcc -fno-strict-aliasing -Wsign-compare -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch i386 -arch x86_64 -g -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c plyvel/_plyvel.cpp -o build/temp.macosx-10.6-intel-3.7/plyvel/_plyvel.o -Wall -g -x c++ -std=c++11 -stdlib=libc++ clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) clang: error: invalid deployment target for -stdlib=libc++ (requires OS X 10.7 or later) error: command 'gcc' failed with exit status 1 ----------------------------------------
I obviously have an OSX later than 10.7, but for some reason this is not recognised. I would be very grateful for any help concerning this issue.
Best Pia
i don't use osx myself, sorry 😕
that said, there are a few seemingly similar (closed) issues in the issue tracker. maybe you can have a look, and if that doesn't help, gently ping some more knowledgeable people in this issue.
I met one similar issue and have solved it by installing leveldb first.
brew install leveldb
pip install plyvel
In my case I was not able to install plyvel in a virtual environment that was created from the macOS provided Python 3.8 installation. pip install plyvel was giving me all sorts of compilation errors. As soon as I told virtualenv to create the virtual environment from the brew provided Python 3.8, plyvel installed without any issues.
Try:
brew install leveldb
brew install [email protected]
virtualenv --python /usr/local/opt/[email protected]/bin/python3 venv
source venv/bin/activate
pip install plyvel
Update:
Spoke too soon. Installation went fine, however when I try to run a Python script that uses plyvel I get: Symbol not found: __ZTIN7leveldb10ComparatorE...
Update:
Spoke too soon. Installation went fine, however when I try to run a Python script that uses
plyvelI get:Symbol not found: __ZTIN7leveldb10ComparatorE...
I'm getting the exact same error.
To give a little more context:
leveldb=1.23
plyvel=1.30
Tried both Python3.8 and Python3.9.
All worked well before updating, and leveldb released a new version 15 days ago, so I'm guessing there's gonna be something there, but cannot install 1.22 from brew again so I've been unable to test it :(
Got it to work by reinstalling leveldb 1.22, so I can confirm the issue is with the latest version.
did you recompile plyvel after upgrading leveldb? if not, i think you should.
I did after upgrading / downgrading. It only work aster downgrading and recompiling.
are you sure it recompiled (and didn't use some cached wheel somewhere on your disk)?
@wbolster looks like that was the issue indeed, it was building using and old cached version.
Sorry for the unnecessary noise.
@sr-gi How did you install leveldb 1.22? It seems brew no longer supports it.
@rickwierenga I had to modify the formula myself to point to the proper version. You can get historical versions by browsing homebrew's GitHub.
Seems to me that a leveldb 1.23 change disabled RTTI, and typeinfo symbols are missing from the built library, whereas they were present in leveldb 1.22.
See related issue https://github.com/google/leveldb/issues/731
Managed to get it to work as well with leveldb version 1.22.
In order to get leveldb 1.22 installed using brew, first I had to create my own Homebrew tap. To get yours created, you can follow the instructions I put in my own Homebrew formulae repo: https://github.com/bagonyi/homebrew-formulae
Once leveldb with the correct version was installed, I uninstalled plyvel and reinstalled it with caching disabled:
pip uninstall plyvel
pip install plyvel --no-cache-dir
Tested that a custom built leveldb 1.23, with the following line removed from its CMakeLists.txt, works fine.
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fno-rtti")
Thanks @bagonyi Your 1.22 leveldb with the following worked for me on Big Sur:
pip uninstall plyvel
CFLAGS='-mmacosx-version-min=11.2 -stdlib=libc++ -std=c++11' pip install plyvel --upgrade
Update: Spoke too soon. Installation went fine, however when I try to run a Python script that uses
plyvelI get:Symbol not found: __ZTIN7leveldb10ComparatorE...I'm getting the exact same error.
To give a little more context:
leveldb=1.23plyvel=1.30Tried both Python3.8 and Python3.9.
All worked well before updating, and leveldb released a new version 15 days ago, so I'm guessing there's gonna be something there, but cannot install 1.22 from brew again so I've been unable to test it :(
Update: Spoke too soon. Installation went fine, however when I try to run a Python script that uses
plyvelI get:Symbol not found: __ZTIN7leveldb10ComparatorE...I'm getting the exact same error.
To give a little more context:
leveldb=1.23plyvel=1.30Tried both Python3.8 and Python3.9.
All worked well before updating, and leveldb released a new version 15 days ago, so I'm guessing there's gonna be something there, but cannot install 1.22 from brew again so I've been unable to test it :(
I also encounter this error on OSX when compiling a static leveldb library libleveldb.a and building plyvel wheel with libleveldb.a. As @CypherGrue said in leveldb 1.23 diff with 1.22, it disables RTTI by add flag -fno-rtti. So when linking a static library libleveldb.a which did not have RTTI, I build _plyvel.cpython-37m-darwin.so also with no RTTI by add same cflags in setup.py.
...
extra_compile_args = ['-Wall', '-g', '-x', 'c++', '-std=c++11', '-fno-rtti']
...
After that, this problem was fixed.
python3 -c "import plyvel; print(plyvel.__leveldb_version__); print(plyvel.__version__)"
1.23
1.3.0
(venv)
In addition, I also built plyvel on Windows. It is fine for msvc compiler when mixing RTTI and no RTTI, so you do not need add /GR- (RTTI disabled) on Windows
Seems like this also occurs when using MacPorts, although the error is different:
/usr/bin/clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -pipe -Os -isysroot/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -arch x86_64 -I/Users/agatti/Library/Caches/pypoetry/virtualenvs/hs-workflows-plgAVo18-py3.8/include -I/opt/local/Library/Frameworks/Python.framework/Versions/3.8/include/python3.8 -c plyvel/_plyvel.cpp -o build/temp.macosx-10.15-x86_64-3.8/plyvel/_plyvel.o -Wall -g -x c++ -std=c++11 -stdlib=libc++
plyvel/_plyvel.cpp:632:10: fatal error: 'leveldb/db.h' file not found
#include "leveldb/db.h"
^~~~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit status 1
Both Python 3.8 and its virtualenv package come from MacPorts so it should pick up the proper include directory (/opt/local/include) when building the package.
Hii, just installed [email protected] following @bagonyi tips and works fine for me
Managed to get it to work as well with
leveldbversion 1.22.In order to get
leveldb1.22 installed usingbrew, first I had to create my own Homebrew tap. To get yours created, you can follow the instructions I put in my own Homebrew formulae repo: https://github.com/bagonyi/homebrew-formulaeOnce
leveldbwith the correct version was installed, I uninstalledplyveland reinstalled it with caching disabled:pip uninstall plyvel pip install plyvel --no-cache-dir
plyvel/_plyvel.cpp:703:10: fatal error: 'leveldb/db.h' file not found
#include "leveldb/db.h"
^~~~~~~~~~~~~~
1 error generated.
error: command '/usr/bin/clang' failed with exit code 1
[end of output]
I'm also getting this error on trying to pip install plyvel or install it from source, with leveldb installed through homebrew, on an M1 (arm) mac
Just solved this by prefixing either the make or the pip install with CFLAGS="-I/opt/homebrew/include -L/opt/homebrew/lib", as suggested by the following stack exchange answer.
https://stackoverflow.com/a/70375604
I don't know if that help, but I succeeded to build it on freebsd with this command:
CFLAGS='-g -stdlib=libc++ -std=c++11 -fno-rtti' pip install --force-reinstall --global-option="build_ext" --global-option="-I/usr/local/include" --global-option="-L/usr/local/lib" plyvel
CFLAGS='-g -stdlib=libc++ -std=c++11 -fno-rtti' pip install --force-reinstall --global-option="build_ext" --global-option="-I/usr/local/include" --global-option="-L/usr/local/lib" plyvel
M1 Mac Monterey. Using this + build leveldb from source @ 1.22.0 help plyvel find leveldb. But it still cannot find snappy. Have tried install snappy from source or brew
ImportError: dlopen(/opt/homebrew/Caskroom/miniforge/base/envs/scraper/lib/python3.10/site-packages/plyvel/_plyvel.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__ZN6snappy11RawCompressEPKcmPcPm'
This worked for me with Monterey and homebrew:
CFLAGS="-I/opt/homebrew/include/ -L/opt/homebrew/lib/ -fno-rtti" pip install --force-reinstall --global-option="build_ext" plyvel
I'm on an M1 machine so you may have to amend the -I and -L argments.
The command given above might not work for everyone, if for whatever reason the Homebrew symlinks are not under /opt/homebrew.
You can get the install directory of any Homebrew package with brew --prefix, so adapting the command to use this dynamic value gives:
CFLAGS="-I$(brew --prefix leveldb)/include/ -L$(brew --prefix leveldb)/lib/ -fno-rtti" pip install --force-reinstall --global-option="build_ext" plyvel
Verify that the library was linked without issues by simply loading the module:
python -c 'import plyvel'
Another approach is switch to MacPorts where I keep RTTI at LevelDB as a variant and by default it builds with it. So, plyvel works out of the box :)
plyvel/_plyvel.cpp:703:10: fatal error: 'leveldb/db.h' file not found #include "leveldb/db.h" ^~~~~~~~~~~~~~ 1 error generated. error: command '/usr/bin/clang' failed with exit code 1 [end of output]I'm also getting this error on trying to pip install plyvel or install it from source, with leveldb installed through homebrew, on an M1 (arm) mac
Also happening from M2 (arm) mac... I already tried with [email protected] version and all flags specified in this thread.
This worked for me on M1 / 14.3 with simple brew install leveldb
CFLAGS="-I/opt/homebrew/include -w" LDFLAGS="-L/opt/homebrew/lib" pip install plyvel
CFLAGS='-g -stdlib=libc++ -std=c++11 -fno-rtti' pip install --force-reinstall --global-option="build_ext" --global-option="-I/usr/local/include" --global-option="-L/usr/local/lib" plyvel
M1 Mac Monterey. Using this + build leveldb from source @ 1.22.0 help
plyvelfindleveldb. But it still cannot findsnappy. Have tried install snappy from source or brewImportError: dlopen(/opt/homebrew/Caskroom/miniforge/base/envs/scraper/lib/python3.10/site-packages/plyvel/_plyvel.cpython-310-darwin.so, 0x0002): symbol not found in flat namespace '__ZN6snappy11RawCompressEPKcmPcPm'
I was running into the same issue as @trungtin, and solved it by building snappy and then leveldb from source with cmake -DBUILD_SHARED_LIBS=ON. I also had to add export DYLD_LIBRARY_PATH="$DYLD_LIBRARY_PATH:/usr/local/lib/" to make sure pylvel could find them.
Thanks for all the suggestions in this thread!
This works for me on MacOs Sonoma M2 Pro
CFLAGS="-I$(brew --prefix leveldb)/include/ -L$(brew --prefix leveldb)/lib/ -fno-rtti" pip install --force-reinstall --config-settings='--global-option="build_ext"' --no-cache-dir plyvel
