build fails with python 3.9(.1) on latest rpi (buster)

[sebres]

Despite the docu says that slot tp_print was already renamed in 3.8, but it looks like a certain incompatibility using python 3.9 only (missing some rule/definition in make file?), because it still seemed to use these "obsolete" members for some reason:

$ python3 --version
Python 3.9.1
$ python3 -m pip install python-openzwave |& tee log.txt
Collecting python-openzwave
  Using cached python_openzwave-0.4.19.zip (147 kB)
Requirement already satisfied: pyserial in ./lib/python3.9/site-packages (from python-openzwave) (3.5)
...
    Running setup.py install for python-openzwave: finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'.../setup.py'"'"'; __file__='"'"'.../setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record .../install-record.txt --single-version-externally-managed --compile --install-headers /ha/include/site/python3.9/python-openzwave
    Complete output (1689 lines):
    ImportError in : from wheel.bdist_wheel import bdist_wheel as _bdist_wheel
    sysargv ['.../setup.py', 'install', '--record', '/tmp/pip-record-kf__8sp5/install-record.txt', '--single-version-externally-managed', '--compile', '--install-headers', '/ha/include/site/python3.9/python-openzwave']
    Found SETUP_DIR : ...
    NameError in : class bdist_wheel(_bdist_wheel) - Use bdist_egg instead
    <pyozw_setup.EmbedTemplate object at 0xb5a33f88>
    {'name': 'libopenzwave', 'sources': ['openzwave-embed/open-zwave-master/python-openzwave/src-lib/libopenzwave/libopenzwave.cpp'], 'include_dirs': ['src-lib/libopenzwave/', 'openzwave-embed/open-zwave-master/cpp/src', 'openzwave-embed/open-zwave-master/cpp/src/value_classes', 'openzwave-embed/open-zwave-master/cpp/src/platform', 'openzwave-embed/open-zwave-master/cpp/build/linux'], 'define_macros': [('PY_LIB_VERSION', '0.4.19'), ('PY_SSIZE_T_CLEAN', 1), ('PY_LIB_FLAVOR', 'embed'), ('PY_LIB_BACKEND', 'cpp')], 'libraries': ['udev', 'stdc++', 'resolv'], 'extra_objects': ['openzwave-embed/open-zwave-master/libopenzwave.a'], 'extra_compile_args': [], 'extra_link_args': [], 'language': 'c++'}
    ['six', 'pyserial', 'PyDispatcher>=2.0.5']
    running install
...
    .../src-lib/libopenzwave/libopenzwave.cpp: In function ‘int __Pyx_modinit_type_init_code()’:
    .../src-lib/libopenzwave/libopenzwave.cpp:40702:39: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’; did you mean ‘tp_dict’?
       __pyx_type_12libopenzwave_PyOptions.tp_print = 0;
                                           ^~~~~~~~
                                           tp_dict
    .../src-lib/libopenzwave/libopenzwave.cpp:40720:38: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’; did you mean ‘tp_dict’?
       __pyx_type_12libopenzwave_RetAlloc.tp_print = 0;
                                          ^~~~~~~~
                                          tp_dict
    .../src-lib/libopenzwave/libopenzwave.cpp:40728:54: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’; did you mean ‘tp_dict’?
       __pyx_type_12libopenzwave_InstanceAssociationAlloc.tp_print = 0;
                                                          ^~~~~~~~
                                                          tp_dict
    .../src-lib/libopenzwave/libopenzwave.cpp:40736:39: error: ‘PyTypeObject’ {aka ‘struct _typeobject’} has no member named ‘tp_print’; did you mean ‘tp_dict’?
       __pyx_type_12libopenzwave_PyManager.tp_print = 0;
                                           ^~~~~~~~
                                           tp_dict
...
    error: command '/usr/bin/gcc' failed with exit code 1

Build with python 3.8.7 succeeds without any error (no ImportError and no gcc errors at all).

[king-dopey]

This isn't just on Raspberry Pi. One of my builds has been failing for about 3 months, due to the promotion of python 3.9, on Docker Hub. I had to temporarily pin it to Python 3.8, to receive mainstream updates (like security updates).

Given the status of this repo, it may not be a good idea to wait for this to get fixed. This cause is documented here: https://docs.python.org/3/whatsnew/3.9.html

The tp_print slot of PyTypeObject has been removed. It was used for printing objects to files in Python 2.7 and before. Since Python 3.0, it has been ignored and unused. (Contributed by Jeroen Demeyer in bpo-36974.)

And here: https://bugs.python.org/issue37250

C files generated by Cython set tp_print to 0: PyTypeObject.tp_print removed

A cython rebuild, using updated openzwave source, may even solve this, for Python 3.9.

[king-dopey]

LOL, I had forgotten that I worked around this, back in July, on my OpenZWave docker images (forked from the official images, which don't work anymore). I did this back in July.... The workaround was exactly as I stated. By switching to the Git flavor, everything is fully rebuilt and works just fine with Python 3.9. There are full examples of images being successfully built daily on GitHub.. and the binaries on DockerHub. Hope this helps others. I should update my Pytomation image, as another example, soonish.

[leonarddevries]

Hi, I also had problems compiling python-openzwave==0.4.19 for python 3.9. For me it was solved by using Cython==0.29.14 instead of 0.28.6. I'm not sure if changing the Cython version might cause any other problems, I'll test that in the coming weeks.

See the changelog of Cython mentioning the 'tp_print' fix.

[mvonrose]

I've been getting a similar error that is prefaced by:

  ERROR: Failed building wheel for Cython
  Running setup.py clean for Cython
Successfully built python-openzwave
Failed to build Cython
Installing collected packages: Cython, python-openzwave
  Attempting uninstall: Cython
    Found existing installation: Cython 0.29.26
    Uninstalling Cython-0.29.26:
      Successfully uninstalled Cython-0.29.26
    Running setup.py install for Cython ... error

and then finally:

ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-q0b8u3s8/cython_3dcf4e0d56204663b318251ade45c314/setup.py'"'"'; __file__='"'"'/tmp/pip-install-q0b8u3s8/cython_3dcf4e0d56204663b318251ade45c314/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ovotu_j2/install-record.txt --single-version-externally-managed --user --prefix= --compile --install-headers /home/pi/.local/include/python3.9/Cython Check the logs for full command output.

@leonarddevries you mentioned compling using Cython==0.29.14 solved this? I'm a bit out of my depth here, and just wondering if that's something I can do using pip? Or I need to compile from source and install with make?

[pmarks-net]

Or I need to compile from source and install with make?

This works for me, on Raspberry Pi OS 11:

sudo apt install -y cython3 libopenzwave1.6-dev
git clone https://github.com/OpenZWave/python-openzwave.git
cd python-openzwave
sed -i 's/Cython==0.28.6/Cython>=0.29/' pyozw_setup.py
./setup.py install --user --flavor=shared

[kdschlosser]

for those people that wan use ZWave with Python I just made library that will do it. It still uses OpenZwave and is to some degree based on python-openzwave. There are breaking changes to the API but the benefits were too large to not break the API. The speed increases are enormous and I added support for multi instance devices and association groups. Once you get the library up and running how you want I coded in the ability to compile the entire library into C Extensions for even more of a speed boost. compiling the library into extensions I would do after everything is up and running otherwise all of the nifty IDE features will not work because I have not created stub files for all of the modules.

I am still working on the build system and within the next few days I will publish wheels to Pypi for OSX11, OSX12, Windows 10, Windows 11, Ubuntu 20.04 and Ubuntu 22.04 running Python 3.8, 3.9 and 3.10. I will be adding NetBSD, FreeBSD and SunOS to that list down the road.

Everything is documented and I am still working on building the documentation. I hope to have that done within the next week as well.

https://github.com/kdschlosser/libopenzwave

[SchulerSimon]

The solution from @pmarks-net worked for me with ubuntu22.04 and its supported python version python3.10.