pyfst
pyfst copied to clipboard
vchahun
pip install pyfst fails on OSX Yosemite
Strange that it can't seem to find the unordered_map header. Any suggestions?
gcc -fno-strict-aliasing -I//anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I//anaconda/include/python2.7 -c fst/_fst.cpp -o build/temp.macosx-10.5-x86_64-2.7/fst/_fst.o -O2
In file included from fst/_fst.cpp:321:
/Users/delip/prefix/include/fst/symbol-table.h:26:10: fatal error: 'unordered_map' file not found
#include <unordered_map>
^
1 error generated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Failed building wheel for pyfst
Failed to build pyfst
Installing collected packages: pyfst
Running setup.py install for pyfst
Complete output from command //anaconda/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/qj/c2h005w56nlc2hpc5n1f8s5r0000gn/T/pip-build-UETMSM/pyfst/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/qj/c2h005w56nlc2hpc5n1f8s5r0000gn/T/pip-cq1YQe-record/install-record.txt --single-version-externally-managed --compile:
running install
running build
running build_py
running build_ext
building 'fst._fst' extension
gcc -fno-strict-aliasing -I//anaconda/include -arch x86_64 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -I//anaconda/include/python2.7 -c fst/_fst.cpp -o build/temp.macosx-10.5-x86_64-2.7/fst/_fst.o -O2
In file included from fst/_fst.cpp:321:
/Users/delip/prefix/include/fst/symbol-table.h:26:10: fatal error: 'unordered_map' file not found
#include <unordered_map>
^
1 error generated.
error: command 'gcc' failed with exit status 1
----------------------------------------
Command "//anaconda/bin/python -c "import setuptools, tokenize;__file__='/private/var/folders/qj/c2h005w56nlc2hpc5n1f8s5r0000gn/T/pip-build-UETMSM/pyfst/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/qj/c2h005w56nlc2hpc5n1f8s5r0000gn/T/pip-cq1YQe-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/qj/c2h005w56nlc2hpc5n1f8s5r0000gn/T/pip-build-UETMSM/pyfst
I just ran into this issue too, from what I can tell this is a compiler error, and you need to specify the use of a newer compiler (which you can install with homebrew).
I made some progress on this issue, but it's still not fixed.
with homebrew install gcc4.7
brew tap homebrew/versions
brew install gcc47
Inside the setup.py file after the imports of modules add the following:
os.environ["CC"] = "gcc-4.7"
os.environ["CXX"] = "g++-4.7"
It now compiles/builds, however it still isn't functioning...
can you expand on "it still isn't functioning.." ?
On Mon, Oct 12, 2015 at 2:42 AM, Ogi Moore [email protected] wrote:
I made some progress on this issue, but it's still not fixed.
with homebrew install gcc4.7
brew tap homebrew/versions brew install gcc47
Inside the setup.py file after the imports of modules add the following:
os.environ["CC"] = "gcc-4.7" os.environ["CXX"] = "g++-4.7"
It now compiles/builds, however it still isn't functioning...
— Reply to this email directly or view it on GitHub https://github.com/vchahun/pyfst/issues/24#issuecomment-147301011.
After the compile finishes, I start python, and upon running import fst, I get errors about symbols missing from _fst.so
Sorry I can't give a more accurate error, I'm at work right now, I'll follow up when I get home.
Here is an example error I get after a successful compile of pyfst:
ogi@Mac-VM ~ ❯❯❯ ipython
Python 3.4.3 |Continuum Analytics, Inc.| (default, Mar 6 2015, 12:07:41)
Type "copyright", "credits" or "license" for more information.
IPython 4.0.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import fst
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
<ipython-input-1-7279038b919c> in <module>()
----> 1 import fst
/Users/ogi/miniconda3/lib/python3.4/site-packages/pyfst-0.2.3.dev0-py3.4-macosx- 10.5-x86_64.egg/fst/__init__.py in <module>()
----> 1 from fst._fst import EPSILON, EPSILON_ID, SymbolTable,\
2 read, read_log, read_std, read_symbols, \
3 LogWeight, LogArc, LogState, LogVectorFst,\
4 TropicalWeight, StdArc, StdState, StdVectorFst, read_symbols_tex t
5
ImportError: dlopen(/Users/ogi/miniconda3/lib/python3.4/site-packages/pyfst-0.2. 3.dev0-py3.4-macosx-10.5-x86_64.egg/fst/_fst.so, 2): Symbol not found: __ZTINSt8 ios_base7failureB5cxx11E
Referenced from: /Users/ogi/miniconda3/lib/python3.4/site-packages/pyfst-0.2.3 .dev0-py3.4-macosx-10.5-x86_64.egg/fst/_fst.so
Expected in: dynamic lookup
I can successfully build with later versions of gcc, but I get errors similar to this one.
Suggestions?
EDIT:
I should point out I'm using the UFAL-DSG fork of pyfst as it has been more recently updated; however development has stopped there as well.
I'm guessing this is because of a Python version mismatch. I see you have Python 3.4. Can you try with Python 2.7.x?
On Mon, Oct 12, 2015 at 7:30 PM, Ogi Moore [email protected] wrote:
Here is an example error I get after a successful compile of pyfst:
ogi@Mac-VM ~ ❯❯❯ ipython Python 3.4.3 |Continuum Analytics, Inc.| (default, Mar 6 2015, 12:07:41) Type "copyright", "credits" or "license" for more information.
IPython 4.0.0 -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details.
In [1]: import fst
ImportError Traceback (most recent call last)
in () ----> 1 import fst /Users/ogi/miniconda3/lib/python3.4/site-packages/pyfst-0.2.3.dev0-py3.4-macosx- 10.5-x86_64.egg/fst/init.py in
() ----> 1 from fst._fst import EPSILON, EPSILON_ID, SymbolTable,
2 read, read_log, read_std, read_symbols,
3 LogWeight, LogArc, LogState, LogVectorFst,
4 TropicalWeight, StdArc, StdState, StdVectorFst, read_symbols_tex t 5ImportError: dlopen(/Users/ogi/miniconda3/lib/python3.4/site-packages/pyfst-0.2. 3.dev0-py3.4-macosx-10.5-x86_64.egg/fst/_fst.so, 2): Symbol not found: __ZTINSt8 ios_base7failureB5cxx11E Referenced from: /Users/ogi/miniconda3/lib/python3.4/site-packages/pyfst-0.2.3 .dev0-py3.4-macosx-10.5-x86_64.egg/fst/_fst.so Expected in: dynamic lookup
I can successfully build with later versions of gcc, but I get errors similar to this one.
Suggestions?
— Reply to this email directly or view it on GitHub https://github.com/vchahun/pyfst/issues/24#issuecomment-147548503.
I can try it later; however I do have pyfst working* on on my linux system with python 3. Again, note that I am using this fork: https://github.com/UFAL-DSG/pyfst of pyfst (not this one).
** for some reason the plotting within the jupyter notebook doesn't work with python 3, but works with python 2.
I got it working!
there are a number of steps to be done here, not sure if they're all necessary but this is what I did
- brew uninstall openfst
- brew install gcc49
- brew install openfst --build-from-source --cc=gcc-4.9 -cxx=g++-4.9
- git clone https://github.com/UFAL-DSG/pyfst.git
- nano ./pyfst/setup.py
⋅⋅1. near the top of the file, after the imports, add the following two lines of code:
os.environ["CC"] = "gcc-4.9"
os.environ["CXX"] = "g++-4.9"
- set the environmental variables as described here: https://github.com/UFAL-DSG/pyfst
- sudo python ./pyfst/setup.py install
- sudo mv /usr/lib/libstdc++.6.dylib libstdc++.6.dylib.old
- sudo cp /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/libstdc++.6.dylib /usr/lib/libstdc++.6.dylib
you will then be able to start a python kernel and import fst
This is great. Well done! Pyfst is notoriously hard to install. It will be great to turn this into a docker image.
On Wed, Oct 14, 2015 at 1:47 AM, Ogi Moore [email protected] wrote:
I got it working!
there are a number of steps to be done here, not sure if they're all necessary but this is what I did
- brew uninstall openfst 2, brew install gcc49 3, brew install openfst --build-from-source --cc=gcc-4.9 -cxx=g++-4.9
- git clone https://github.com/UFAL-DSG/pyfst.git
- nano ./pyfst/setup.py ⋅⋅*near the top of the file, after the imports, add the following two lines of code:
os.environ["CC"] = "gcc-4.9" os.environ["CXX"] = "g++-4.9"
- set the environmental variables as described here: https://github.com/UFAL-DSG/pyfst
- sudo python ./pyfst/setup.py install
- sudo mv /usr/lib/libstdc++.6.dylib libstdc++.6.dylib.old
- sudo cp /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/libstdc++.6.dylib /usr/lib/libstdc++.6.dylib
you will then be able to start a python kernel and import fst
— Reply to this email directly or view it on GitHub https://github.com/vchahun/pyfst/issues/24#issuecomment-147942242.
Not sure how a python library would work in a docker image on its own,but I'm not a docker pro so I would be the last person to know how well that would work.
There is still the IPython/Jupyter plotting issue that occurs when running pyfst in Python 3. Looking through the code with the debugger, it's clear that I'm in over my head. On Oct 15, 2015 10:13 AM, "Delip Rao" [email protected] wrote:
This is great. Well done! Pyfst is notoriously hard to install. It will be great to turn this into a docker image.
On Wed, Oct 14, 2015 at 1:47 AM, Ogi Moore [email protected] wrote:
I got it working!
there are a number of steps to be done here, not sure if they're all necessary but this is what I did
- brew uninstall openfst 2, brew install gcc49 3, brew install openfst --build-from-source --cc=gcc-4.9 -cxx=g++-4.9
- git clone https://github.com/UFAL-DSG/pyfst.git
- nano ./pyfst/setup.py ⋅⋅*near the top of the file, after the imports, add the following two lines of code:
os.environ["CC"] = "gcc-4.9" os.environ["CXX"] = "g++-4.9"
- set the environmental variables as described here: https://github.com/UFAL-DSG/pyfst
- sudo python ./pyfst/setup.py install
- sudo mv /usr/lib/libstdc++.6.dylib libstdc++.6.dylib.old
- sudo cp /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/libstdc++.6.dylib /usr/lib/libstdc++.6.dylib
you will then be able to start a python kernel and import fst
— Reply to this email directly or view it on GitHub https://github.com/vchahun/pyfst/issues/24#issuecomment-147942242.
— Reply to this email directly or view it on GitHub https://github.com/vchahun/pyfst/issues/24#issuecomment-148461732.
My only worry is steps 3&4.
- sudo mv /usr/lib/libstdc++.6.dylib libstdc++.6.dylib.old
- sudo cp /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/libstdc++.6.dylib /usr/lib/libstdc++.6.dylib
This could cause other builds to fail. Did you try using LD_LIBRARY_PATH instead of changing these files?
On Thu, Oct 15, 2015 at 1:12 PM, Delip Rao [email protected] wrote:
This is great. Well done! Pyfst is notoriously hard to install. It will be great to turn this into a docker image.
On Wed, Oct 14, 2015 at 1:47 AM, Ogi Moore [email protected] wrote:
I got it working!
there are a number of steps to be done here, not sure if they're all necessary but this is what I did
- brew uninstall openfst 2, brew install gcc49 3, brew install openfst --build-from-source --cc=gcc-4.9 -cxx=g++-4.9
- git clone https://github.com/UFAL-DSG/pyfst.git
- nano ./pyfst/setup.py ⋅⋅*near the top of the file, after the imports, add the following two lines of code:
os.environ["CC"] = "gcc-4.9" os.environ["CXX"] = "g++-4.9"
- set the environmental variables as described here: https://github.com/UFAL-DSG/pyfst
- sudo python ./pyfst/setup.py install
- sudo mv /usr/lib/libstdc++.6.dylib libstdc++.6.dylib.old
- sudo cp /usr/local/Cellar/gcc49/4.9.3/lib/gcc/4.9/libstdc++.6.dylib /usr/lib/libstdc++.6.dylib
you will then be able to start a python kernel and import fst
— Reply to this email directly or view it on GitHub https://github.com/vchahun/pyfst/issues/24#issuecomment-147942242.
So after talking with one of the developers for openfst, it turns out that openfst now (as of 1.5.0) supports python bindings, you just need to add --enable-python in the compile options.
Alternatively, you can perform a "pip install openfst" as well; however a lot of the same issues we've encountered here (won't compile, symbol not found, etc) are still occurring.
I'm shifting my efforts to getting the openfst python bindings working as pyfst development seems to have been abandoned.