python-soundfile icon indicating copy to clipboard operation
python-soundfile copied to clipboard

Published 20 hours ago • verified publisher icon bastibe

why libsndfile is not packed into the Linux wheels

Open polinaeterna opened this issue 2 years ago • 8 comments

Hi! What is the reason for libsndfile system library not being packed into the wheels for Linux, as it is done for Mac and Windows? We'd like to use new pysoundfile mp3 support in the Hugging Face Datasets library, but currently it requires building libsndfile==1.1.0 from source as this version cannot be installed via a package manager. And building libsndfile from source is not super convenient for most of the users.

polinaeterna avatar Oct 20 '22 16:10 polinaeterna

@bastibe I probably misunderstood the idea before, for Mac and Windows you compile libsndfile binaries yourself and install them separately along with with pysoundfile library, right? Isn't it possible to just put it in the python wheel prebuilt?

polinaeterna avatar Oct 21 '22 11:10 polinaeterna

As far as I'm aware, building cross-distribution libraries is very easy to get wrong. If you show me how to build the libraries reliably and portably on Linux, I'll gladly add those binary wheels.

bastibe avatar Oct 21 '22 12:10 bastibe

Maybe manylinux can help ? They have an example repository here to build wheels for many linux distros at the same time

https://github.com/pypa/python-manylinux-demo

lhoestq avatar Oct 21 '22 13:10 lhoestq

I've found some explanation and instructions here: https://realpython.com/python-wheels/#bundling-shared-libraries As far as I understand, this is the way to put C binaries inside a python wheel for more or less any Linux variant, so that users don't need to bother with their installation via a package manager themselves.

polinaeterna avatar Oct 21 '22 13:10 polinaeterna

Fascinating! Thank you! I'll have a look.

If you'd like to take a stab at it, I'd be extremely grateful for a pull request to https://github.com/bastibe/libsndfile-binaries/, either with pre-built libsndfile binaries, or better yet, with a CI script for generating them.

bastibe avatar Oct 22 '22 08:10 bastibe

I am definitely not an expert in this area but isn't this a case where cibuildwheel could help?
https://cibuildwheel.readthedocs.io/en/stable/#usage

Many examples here: https://cibuildwheel.readthedocs.io/en/stable/working-examples/

NickleDave avatar Oct 29 '22 10:10 NickleDave

Just so you know, I probably won't have much time to devote to this in the near future. So if anyone wants to take a stab at the CI script over at https://github.com/bastibe/libsndfile-binaries/, I'd be more than grateful for a pull request.

bastibe avatar Oct 30 '22 11:10 bastibe

Work for this is ongoing in https://github.com/bastibe/libsndfile-binaries/pull/17

bastibe avatar Nov 04 '22 10:11 bastibe