pynini
pynini copied to clipboard
Published
20 hours ago •
kylebgorman
kylebgorman
pip install fails on Mac OS 13.5.1
Hi Kyle! I'm trying to install Pynini 2.1.4. I'm working on something that requires OpenFST 1.8.1, so 2.1.5 is out (though I can install 2.1.5 fine from conda-forge). I'm running into difficulty with pip install pynini==2.1.4. I don't see a way to install 2.1.4 from conda-forge.
This (pip installing 2.1.4) used to work on our M1 macs, so I'm wondering if something has changed in Cython or elsewhere that causes this issue.
(slu) ➜ pynini-2.1.4 pip install pynini==2.1.4
Collecting pynini==2.1.4
Using cached pynini-2.1.4.tar.gz (621 kB)
Preparing metadata (setup.py) ... done
Requirement already satisfied: Cython>=0.29 in /Users/Simon.Charlow/opt/miniconda3/envs/slu/lib/python3.8/site-packages (from pynini==2.1.4) (3.0.2)
Building wheels for collected packages: pynini
Building wheel for pynini (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> [117 lines of output]
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-11.0-arm64-cpython-38
creating build/lib.macosx-11.0-arm64-cpython-38/pywrapfst
copying pywrapfst/__init__.py -> build/lib.macosx-11.0-arm64-cpython-38/pywrapfst
creating build/lib.macosx-11.0-arm64-cpython-38/pynini
copying pynini/__init__.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini
creating build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/chatspeak.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/chatspeak_model.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/weather.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/__init__.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/numbers.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/plurals.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/t9.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/case.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/dates.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/examples/g2p.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
creating build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/edit_transducer.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/utf8.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/__init__.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/features.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/pynutil.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/paradigms.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/tagger.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/rule_cascade.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/rewrite.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/lib/byte.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
creating build/lib.macosx-11.0-arm64-cpython-38/pynini/export
copying pynini/export/__init__.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/export
copying pynini/export/export.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/export
copying pynini/export/multi_grm_example.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/export
copying pynini/export/multi_grm.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/export
copying pynini/export/grm.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/export
copying pynini/export/grm_example.py -> build/lib.macosx-11.0-arm64-cpython-38/pynini/export
copying pywrapfst/__init__.pyi -> build/lib.macosx-11.0-arm64-cpython-38/pywrapfst
copying pywrapfst/py.typed -> build/lib.macosx-11.0-arm64-cpython-38/pywrapfst
copying pynini/__init__.pyi -> build/lib.macosx-11.0-arm64-cpython-38/pynini
copying pynini/py.typed -> build/lib.macosx-11.0-arm64-cpython-38/pynini
copying pynini/examples/py.typed -> build/lib.macosx-11.0-arm64-cpython-38/pynini/examples
copying pynini/lib/py.typed -> build/lib.macosx-11.0-arm64-cpython-38/pynini/lib
copying pynini/export/py.typed -> build/lib.macosx-11.0-arm64-cpython-38/pynini/export
running build_ext
building '_pywrapfst' extension
creating build/temp.macosx-11.0-arm64-cpython-38
creating build/temp.macosx-11.0-arm64-cpython-38/extensions
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /Users/Simon.Charlow/opt/miniconda3/envs/slu/include -arch arm64 -fPIC -O2 -isystem /Users/Simon.Charlow/opt/miniconda3/envs/slu/include -arch arm64 -I/Users/Simon.Charlow/opt/miniconda3/envs/slu/include/python3.8 -c extensions/_pywrapfst.cpp -o build/temp.macosx-11.0-arm64-cpython-38/extensions/_pywrapfst.o -std=c++17 -Wno-register -Wno-deprecated-declarations -Wno-unused-function -Wno-unused-local-typedefs -funsigned-char -stdlib=libc++ -mmacosx-version-min=10.7
In file included from extensions/_pywrapfst.cpp:1220:
In file included from /Users/Simon.Charlow/work/slu/fst_tools/include/fst/fstlib.h:48:
/Users/Simon.Charlow/work/slu/fst_tools/include/fst/const-fst.h:405:25: warning: comparison of integers of different signs: 'int64' (aka 'long long') and 'size_t' (aka 'unsigned long') [-Wsign-compare]
if (hdr.NumStates() != num_states) {
~~~~~~~~~~~~~~~ ^ ~~~~~~~~~~
/Users/Simon.Charlow/work/slu/fst_tools/include/fst/const-fst.h:409:23: warning: comparison of integers of different signs: 'int64' (aka 'long long') and 'size_t' (aka 'unsigned long') [-Wsign-compare]
if (hdr.NumArcs() != num_arcs) {
~~~~~~~~~~~~~ ^ ~~~~~~~~
extensions/_pywrapfst.cpp:11274:40: error: cannot cast from type 'std::string' (aka 'basic_string<char, char_traits<char>, allocator<char>>') to pointer type 'PyObject *' (aka '_object *')
__pyx_t_2 = __Pyx_PyObject_AsDouble(((PyObject *)__pyx_t_1)); if (unlikely(__pyx_t_2 == ((double)((double)-1)) && PyErr_Occurred())) __PYX_ERR(0, 436, __pyx_L1_error)
^~~~~~~~~~~~~~~~~~~~~
extensions/_pywrapfst.cpp:3302:29: note: expanded from macro '__Pyx_PyObject_AsDouble'
((likely(PyFloat_CheckExact(obj))) ? PyFloat_AS_DOUBLE(obj) :\
^~~
/Users/Simon.Charlow/opt/miniconda3/envs/slu/include/python3.8/floatobject.h:24:41: note: expanded from macro 'PyFloat_CheckExact'
#define PyFloat_CheckExact(op) (Py_TYPE(op) == &PyFloat_Type)
^~
/Users/Simon.Charlow/opt/miniconda3/envs/slu/include/python3.8/object.h:122:49: note: expanded from macro 'Py_TYPE'
#define Py_TYPE(ob) (_PyObject_CAST(ob)->ob_type)
^~
/Users/Simon.Charlow/opt/miniconda3/envs/slu/include/python3.8/object.h:111:41: note: expanded from macro '_PyObject_CAST'
#define _PyObject_CAST(op) ((PyObject*)(op))
^~
extensions/_pywrapfst.cpp:1490:43: note: expanded from macro 'likely'
#define likely(x) __builtin_expect(!!(x), 1)
^
extensions/_pywrapfst.cpp:11274:40: error: cannot cast from type 'std::string' (aka 'basic_string<char, char_traits<char>, allocator<char>>') to pointer type 'PyObject *' (aka '_object *')
__pyx_t_2 = __Pyx_PyObject_AsDouble(((PyObject *)__pyx_t_1)); if (unlikely(__pyx_t_2 == ((double)((double)-1)) && PyErr_Occurred())) __PYX_ERR(0, 436, __pyx_L1_error)
^~~~~~~~~~~~~~~~~~~~~
extensions/_pywrapfst.cpp:3302:57: note: expanded from macro '__Pyx_PyObject_AsDouble'
((likely(PyFloat_CheckExact(obj))) ? PyFloat_AS_DOUBLE(obj) :\
^~~
/Users/Simon.Charlow/opt/miniconda3/envs/slu/include/python3.8/floatobject.h:51:51: note: expanded from macro 'PyFloat_AS_DOUBLE'
#define PyFloat_AS_DOUBLE(op) (((PyFloatObject *)(op))->ob_fval)
^~
extensions/_pywrapfst.cpp:11274:40: error: cannot cast from type 'std::string' (aka 'basic_string<char, char_traits<char>, allocator<char>>') to pointer type 'PyObject *' (aka '_object *')
__pyx_t_2 = __Pyx_PyObject_AsDouble(((PyObject *)__pyx_t_1)); if (unlikely(__pyx_t_2 == ((double)((double)-1)) && PyErr_Occurred())) __PYX_ERR(0, 436, __pyx_L1_error)
^~~~~~~~~~~~~~~~~~~~~
extensions/_pywrapfst.cpp:3303:27: note: expanded from macro '__Pyx_PyObject_AsDouble'
likely(PyLong_CheckExact(obj)) ?\
^~~
/Users/Simon.Charlow/opt/miniconda3/envs/slu/include/python3.8/longobject.h:16:40: note: expanded from macro 'PyLong_CheckExact'
#define PyLong_CheckExact(op) (Py_TYPE(op) == &PyLong_Type)
^~
/Users/Simon.Charlow/opt/miniconda3/envs/slu/include/python3.8/object.h:122:49: note: expanded from macro 'Py_TYPE'
#define Py_TYPE(ob) (_PyObject_CAST(ob)->ob_type)
^~
/Users/Simon.Charlow/opt/miniconda3/envs/slu/include/python3.8/object.h:111:41: note: expanded from macro '_PyObject_CAST'
#define _PyObject_CAST(op) ((PyObject*)(op))
^~
extensions/_pywrapfst.cpp:1490:43: note: expanded from macro 'likely'
#define likely(x) __builtin_expect(!!(x), 1)
^
extensions/_pywrapfst.cpp:11274:40: error: cannot cast from type 'std::string' (aka 'basic_string<char, char_traits<char>, allocator<char>>') to pointer type 'PyObject *' (aka '_object *')
__pyx_t_2 = __Pyx_PyObject_AsDouble(((PyObject *)__pyx_t_1)); if (unlikely(__pyx_t_2 == ((double)((double)-1)) && PyErr_Occurred())) __PYX_ERR(0, 436, __pyx_L1_error)
^~~~~~~~~~~~~~~~~~~~~
extensions/_pywrapfst.cpp:3304:18: note: expanded from macro '__Pyx_PyObject_AsDouble'
PyLong_AsDouble(obj) : __Pyx__PyObject_AsDouble(obj))
^~~
extensions/_pywrapfst.cpp:11274:40: error: cannot cast from type 'std::string' (aka 'basic_string<char, char_traits<char>, allocator<char>>') to pointer type 'PyObject *' (aka '_object *')
__pyx_t_2 = __Pyx_PyObject_AsDouble(((PyObject *)__pyx_t_1)); if (unlikely(__pyx_t_2 == ((double)((double)-1)) && PyErr_Occurred())) __PYX_ERR(0, 436, __pyx_L1_error)
^~~~~~~~~~~~~~~~~~~~~
extensions/_pywrapfst.cpp:3304:50: note: expanded from macro '__Pyx_PyObject_AsDouble'
PyLong_AsDouble(obj) : __Pyx__PyObject_AsDouble(obj))
^~~
2 warnings and 5 errors generated.
error: command '/usr/bin/clang' failed with exit code 1
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for pynini
Running setup.py clean for pynini
Failed to build pynini
ERROR: Could not build wheels for pynini, which is required to install pyproject.toml-based projects
I am actually not sure, haven't seen this before, but building (or debugging) from source on a Mac is going to be difficult...hard to get an isolated environment that doesn't have something in the way. However, I did just test this and I believe it'll work on Mac too?
conda create -n simon pynini=2.1.4 -c conda-forge
I don't see a M1-specific version for that release, but I thought it could fall back to the OS X version---let me know if I'm wrong.
FWIW, Pynini 2.1.4 is pretty old, and the file serializations are pretty stable so they can be used across versions.
Thanks for checking, and the quick reply! Unfortunately, it doesn't fall back:
(slu) ➜ ~ conda create -n simon pynini=2.1.4 -c conda-forge
Collecting package metadata (current_repodata.json): done
Solving environment: unsuccessful attempt using repodata from current_repodata.json, retrying with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are not available from current channels:
- pynini=2.1.4
Current channels:
- https://conda.anaconda.org/conda-forge/osx-arm64
- https://conda.anaconda.org/conda-forge/noarch
- https://repo.anaconda.com/pkgs/main/osx-arm64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/osx-arm64
- https://repo.anaconda.com/pkgs/r/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
Yeah, then I'm stuck for easy solutions. I would consider just using Pynini 2.1.5 in one environment and OpenFst 1.8.1 in the other? They can talk to each other through the file system. Is that feasible for your application?
On Conda-Forge's setup, I can always add M1 support to a package that doesn't have it (cf. this), but I don't think there's any way to add M1 support to older versions of packages that didn't have it at the time that version was superceded.
There's definitely things that we can do, just wanted to see if this might be one of them. Thanks again for your quick and clear help.