Native library bundling and CI job for os dependent wheel distribution

[cansik]

As already mentioned in #157 , this PR adds support for native libraries bundled within the package, as well as a multi-os (Windows / Unix) build CI task for github.

At the moment the build task creates wheels for x64 Windows and Linux operating systems and uploads the resulting wheel files into a new release on github. I guess it would make sense to have the following features implemented as well, but it would make sense if the maintainer would define / approve them first:

• Use tag trigger to create a new release by setting a tag
• Add pypi-publish to the build CI to directly upload the binary files to pypi
• Implement an add-all method to directly load all .so files in the package directory (as on Windows) to not be dependent on version numbers
• Maybe re-implement the possibility to use the system Azure libs on Windows (currently I replaced them by the bundled ones)?

[cansik]

@etiennedub I released the bundled library on PyPI: pyk4a-bundle #157

[ZdenekM]

@cansik Great! Just when using pyk4a-bundle, mypy complains for some reason (with "ordinary" pyk4a, it is ok):

src/python/arcor2_kinect_azure/kinect_azure.py:6: error: Skipping analyzing
"pyk4a": module is installed, but missing library stubs or py.typed marker
    import pyk4a

[ZdenekM]

And when trying to open the sensor, I'm unfortunately getting

Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assertion `needed != NULL' failed!

[cansik]

@ZdenekM Thanks for your reports, I will investigate a bit more. I assume you are working on Ubuntu?

I also noticed that currently numpy 1.22.1 is used to compile the wheel packages, which could be a problem for projects with older numpy libraries installed.

[ZdenekM]

Well, I actually tried it within a docker container based on "python:3.9-slim-buster", but on Ubuntu 20.04, I'm getting the same. My numpy is at version 1.22.1.

[shagren]

@cansik , please add pyk4a/py.typed to package_data. It will solve problem.

[cansik]

I have to check why it is not included, I did not change the options in the setup.py, so it should be included as mentioned here: https://github.com/cansik/pyk4a/blob/master/setup.cfg#L23-L25

[cansik]

• Numpy version dependency conflict has been resolved by using the lowest possible numpy version per python version.
• @shagren py.typed should be included again

Released version pyk4a-bundle 1.3.0.1

@ZdenekM I am now working on the linux support, this may take some time

[ZdenekM]

Thank you for your effort!

[cansik]

@ZdenekM I tested the package on a Ubuntu 20.04.3 LTS and at least the color stream seems to work, after installing the device rule. There are still problems with the dynamic loading of the depthengine, but I will work on that.

Update It seems that the auditwheel step only includes the pyk4a and pyk4a-record so lib, but not the depthengine. Manually putting the libdepthengine.so.2.0 into the pyk4a_bundle.libs fixes all issues and the camera works.

I guess the auditwheel step makes copying the libs directly into the module unnecessary for Linux.

[cansik]

After some try-and-error to include the libdepthengine.so.2.0 into the package by auditwheel, I now just use wheel unpack & pack to adjust the wheel after auditing. Definitely not the best way, but it works and the library runs on Linux, also with depth support.

Please try it out by installing version pyk4a-bundle 1.3.0.2

[ZdenekM]

I updated but still getting the same error Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assertion `needed != NULL' failed! (both in docker and on my localhost). The mypy error is gone.

[ZdenekM]

This is the output I got from LD_DEBUG=all python3.9 -c "import pyk4a" - not sure if it could be helpful or not...

pyk4a_ld_debug_all.zip

[ZdenekM]

Maybe this https://github.com/pypa/auditwheel/issues/103 could be related?

[cansik]

@ZdenekM Thanks, which glibc version do you have installed? You can check it by using this command:

ldd --version

[ZdenekM]

ldd (Ubuntu GLIBC 2.31-0ubuntu9.2) 2.31 on my localhost and ldd (Debian GLIBC 2.28-10) 2.28 in the docker image, in which I also tried it.

[codecov[bot]]

Codecov Report

Merging #160 (094ccbf) into develop (27af48d) will decrease coverage by 0.86%. The diff coverage is 18.75%.

@@             Coverage Diff             @@
##           develop     #160      +/-   ##
===========================================
- Coverage    81.38%   80.51%   -0.87%     
===========================================
  Files           12       12              
  Lines          768      780      +12     
===========================================
+ Hits           625      628       +3     
- Misses         143      152       +9     

Impacted Files	Coverage Δ
pyk4a/module.py	25.00% <18.75%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 27af48d...094ccbf. Read the comment docs.

[lpasselin]

@cansik can you share your Dockerfile? It could be useful to publish it in the examples folder.

I will try this new installation on a brand new windows install and report back.

[lpasselin]

On windows, I am getting DLL not found error when I try running the viewer example. Same when manually setting the K4A_DLL_DIR variable to the path to pointing pyk4a folder (containing k4a.dll and depthengine dll).

[cansik]

I will test it this week on more clients, thanks for your feedback.

[cansik]

@lpasselin What kind of error did you get? I am not using any docker images, everything is built on github actions. But I have now create a fresh Windows 11 in hyper-v and installed python 3.8 , pyk4a-bundle and opencv-python. Here the output of the viewer.py:

(venv) PS C:\Users\User\projects\pyk4a> python .\viewer.py
[2022-02-14 16:27:19.399] [error] [t=268] D:\a\1\s\extern\Azure-Kinect-Sensor-SDK\src\usbcommand\usbcommand.c (261): count != 0 returned failure in find_libusb_device()
[2022-02-14 16:27:19.399] [critical] [t=268] D:\a\1\s\extern\Azure-Kinect-Sensor-SDK\src\usbcommand\usbcommand.c (366): find_libusb_device(). libusb device(s) are all unavalable. Is the device being used by another application?
[2022-02-14 16:27:19.400] [error] [t=268] D:\a\1\s\extern\Azure-Kinect-Sensor-SDK\src\depth_mcu\depth_mcu.c (68): usb_cmd_create(USB_DEVICE_DEPTH_PROCESSOR, device_index, NULL, &depthmcu->usb_cmd) returned failure in depthmcu_create()
[2022-02-14 16:27:19.400] [error] [t=268] D:\a\1\s\extern\Azure-Kinect-Sensor-SDK\src\sdk\k4a.c (133): depthmcu_create(index, &device->depthmcu) returned failure in k4a_device_open()
Traceback (most recent call last):
  File ".\viewer.py", line 36, in <module>
    main()
  File ".\viewer.py", line 16, in main
    k4a.start()
  File "C:\Users\User\projects\pyk4a\venv\lib\site-packages\pyk4a\pyk4a.py", line 35, in start
    self.open()
  File "C:\Users\User\projects\pyk4a\venv\lib\site-packages\pyk4a\pyk4a.py", line 67, in open
    self._device_open()
  File "C:\Users\User\projects\pyk4a\venv\lib\site-packages\pyk4a\pyk4a.py", line 84, in _device_open
    _verify_error(res)
  File "C:\Users\User\projects\pyk4a\venv\lib\site-packages\pyk4a\errors.py", line 18, in _verify_error
    raise K4AException()
pyk4a.errors.K4AException

And here the devices.py:

(venv) PS C:\Users\User\projects\pyk4a> python .\devices.py
[2022-02-14 16:30:09.696] [error] [t=3900] D:\a\1\s\extern\Azure-Kinect-Sensor-SDK\src\usbcommand\usbcommand.c (1037): usb_cmd_get_device_count(). No devices found
No devices available

Both seems expected. I currently have problems to passthrough the Azure camera into the hyper-v machine, but there does not seem a library problem.

@ZdenekM I could recreate the issues you have with a fresh 18.04 installation of Ubuntu. I guess there is still something wrong with the manylinux fix. I will further investigate.

[shagren]

@cansik , you can use playback.py for test. It will require both k4a.dll and k4a_record.dll

[lpasselin]

I was having the error triggered by this line.

https://github.com/etiennedub/pyk4a/blob/develop/pyk4a/module.py#L14

[cansik]

@lpasselin Are you sure you have installed the latest pyk4a-bundle version under Windows? I am using it on different Windows computers and can not reconstruct a library error. The link you sent is pointing to the original repository, not my adapted version: pyk4a/module.py

[lpasselin]

@cansik I installed with pip install pyk4a-bundle==1.3.0.2

Will try again today with a different setup.

[lpasselin]

@cansik

regular python 3.9 Works! The k4a SDK must be installed otherwise drivers are not there and Windows does not recognize the device. Installing pyk4a-bundle before installing the drivers works as expected.

anaconda python 3.9 and python 3.8 I tested both 3.6 and 3.9 since the DLL loading is different when using <3.8 and >=3.8 Both result in the same "DLL not found" error mentionned in a previous comment.

Any ideas why the anaconda environment is problematic?

[rajkundu]

I updated but still getting the same error Inconsistency detected by ld.so: dl-version.c: 205: _dl_check_map_versions: Assertion `needed != NULL' failed! (both in docker and on my localhost). The mypy error is gone.

I have the same exact error on Ubuntu 20.04, despite creating brand new conda environments – and even deleting and reinstalling miniconda3, itself, entirely.

[cansik]

@rajkundu Thank you for trying it out. I am aware that Unix is still a problem, maybe someone with many-linux pip package experience can help here. I mainly work on MacOS & Windows, so Linux support was more an additional thing.

@lpasselin Interesting, I am not sure what the problem could be there, but I will give it a try later. Thanks for mentioning!

[rajkundu]

Thank you for your help! Do you have any advice in the meantime for me to be able to get the depth engine working on Linux? For me, the SDK works (no import error) when I simply run pip install pyk4a (no ImportError), but it keeps returning None when I try to do transformations, retrieve depth, etc.

I saw this post:

It seems that the auditwheel step only includes the pyk4a and pyk4a-record so lib, but not the depthengine. Manually putting the libdepthengine.so.2.0 into the pyk4a_bundle.libs fixes all issues and the camera works.

How can I replicate this with just regular pyk4a? I mean, what steps did you take on Linux to get pyk4a working with depth?

[lpasselin]

@rajkundu Does your LD_LIBRARY_PATH contain a directory containing the depthengine.so ? https://github.com/microsoft/Azure-Kinect-Sensor-SDK/blob/develop/docs/depthengine.md#linux