PyYaml missing CSafeLoader

[proximous]

Package name: pyyaml Link to PyPI page: https://pypi.org/project/PyYAML/ Link to piwheels page: https://www.piwheels.org/project/PyYAML/ Version: 5.3.1 Python version: 3.73 I am the maintainer: No More information:

This seems to not be complied with CSafeLoader (meaning libyaml-dev was missing during compile?).

Can the Wheel be rebuilt with the C extensions enabled? It really improves performance.

https://pyyaml.org/wiki/PyYAMLDocumentation

The following classes are available only if you build LibYAML bindings.

CLoader(stream) CSafeLoader(stream) CBaseLoader(stream)

[bennuttall]

Ok - seems reasonable enough. I'll look at doing this soon. I assume it'll require the user to install certain apt packages, and we've got some improvements to how we show them that's to be deployed soon so I'll wait till we've done that.

PyYaml is a very popular package so this will affect a lot of people!

[proximous]

Sounds great. If you compile it yourself you need to apt install libyaml-dev. But you may be able to build it such that libyaml-dev is only needed when you compile, and people who download the wheel don't need to install it. Not sure about that, but it would obviously be better if not too hard.

[proximous]

Any update on this? If I'm not mistaken, I don't think there is a runtime dependency, only a compile-time dependency, meaning I think if you build it with libyaml-dev installed, the wheel will include everything necessary so target systems don't need libyaml-dev.

[geerlingguy]

I ran into this issue on my Pis, after I noticed Ansible running considerably slow, especially on the Pi Zero. To fix for now, I had to run:

sudo apt install -y libyaml-dev  # I had already run this earlier...
sudo pip3 uninstall -y pyyaml
sudo pip3 install --no-cache-dir --no-binary pyyaml pyyaml

[nitzmahone]

pyyaml maintainer here- if you're going to do this, you'll want to make sure you're statically linking libyaml in the extension build, which is a little harder under Linux than it should be. I've prototyped Linux wheel builds in various places, but I've not felt comfortable enough with them to publish them on PyPI. That piwheels is publishing arch-specific pyyaml wheels without the libyaml extension included is definitely a trip-hazard for folks that want it, because even if they have all the bits to build the extension on the pi, it won't get built without --no-binary pyyaml when piwheels is consulted.

Pyyaml has had some folks working on automating the wheel builds for Linux- I added a comment to show how to force the libyaml build to emit a statically linkable artifact here. That PR and the discussion should give you everything necessary to generate a workable libyaml-enabled wheel...

[mi-hol]

@nitzmahone the referenced "automating the wheel builds for Linux" seems merged but @geerlingguy created a blog post making a wide audience aware this issue just yesterday. Could you please comment on below statement from the blog post? "currently the pyyaml wheel for Pi OS / ARM64 is not built correctly to detect libyaml support, so you have to skip the pre-built wheel and basically compile the library yourself. See this issue for more details."

[mjj47]

Hi there!

Any update on this? My company uses bazel to build python packages from wheels. For x86, we pull in pyyaml and use yaml.CSafeLoader all over the place - porting this to aarch64, we hit the woe that CSafeLoader isn't available :(

Is there an updated wheel being published that was statically linked against libyaml and exports CSafe* for pyyaml??

[nitzmahone]

@mjj47 We (PyYAML team, I know nothing about who maintains the piwheels package) have been publishing manylinux_* aarch64 wheels with static libyaml on PyPI for the past several releases. I'm not a frequent enough Pi user to know if PyPI's index takes precedence over piwheels in typical configs, but they're definitely available.

https://pypi.org/project/PyYAML/6.0/#files

[bennuttall]

In the case of e.g. python 3.9 on 64-bit rpios pip will only have a choice between the aarch64 wheel and sdist (both from pypi) - no files provided by piwheels match the combo required in this case - we just have the armv7 wheel.

[bennuttall]

I know nothing about who maintains the piwheels package

piwheels is essentially a pypi mirror - we build armv7 wheels of everything on pypi - no one maintains the individual packages

[mmathys]

Any updates on this?