build icon indicating copy to clipboard operation
build copied to clipboard

How can I change wheel name's platform tag?

Open gregcouch opened this issue 3 years ago • 7 comments

Using "setup.py bdist_wheel", I can change the platform tag used for the binary wheel name from the default linux_x86_64 to manylinux2014_x86_64 by giving bdist_wheel options in setup.py. For example:

setup(..., options={'bdist_wheel': {'plat_name': 'manylinux2014_x86_64'}})

But "python3 -m build" does not use the bdist_wheel options. Is there another way to do it?

For reference, I'm actually picking the most specific platform tag dynamically with:

from packaging import tags
platform = next(tags.cpython_tags()).platform

This is with build==0.8.0, packaging==20.4, setuptools==41.1.0.post, and wheel==0.35.1 with Python 3.6 on CentOS 7.

gregcouch avatar Jun 04 '22 08:06 gregcouch

First, I'd have to ask: are you sure you want to change it? The "correct" way is to produce a linux wheel, then to give it to auditwheel, which will verify it does pass the manylinux spec and bundles (with symbol mangling) any dependencies. Without that, you really do have a "linux" wheel - this means you have no compatibility guarantees and it's not redistributable.

To see how to do this properly, check cibuildwheel, which does all the steps for you.

Anyway, assuming you are aware of that and this is actually what you want, you can see an example of how to change the tags at https://github.com/scikit-build/ninja-python-distributions/pull/85 - though this is not guaranteed to be kept by future versions of wheel. I've got a PR to wheel https://github.com/pypa/wheel/pull/422 that is on hold (wheel development is slow) that would add a utility for renaming.

Finally, I'd guess if you used python -m build --wheel, then you might trigger the standard bdist_wheel call. Without that, it makes an SDist then a wheel from the SDist.

henryiii avatar Jun 04 '22 13:06 henryiii

But auditwheel isn't working for me. When I do the "auditwheel show" is says my wheel 'is consistent with the following platform tag: "manylinux_2_17_x86_64".' But then when I do the "auditwheel repair" it complains that it can not repair 'to "manylinux_2_5_x86_64" ABI because of the presence of too-recent versioned symbols. You'll need to compile the wheel on an older toolchain.'

gregcouch avatar Jun 07 '22 23:06 gregcouch

What are you trying to target? manylinux_2_17_x86_64 (manylinux2014) or manylinux_2_5_x86_64 (manylinux1)? It looks like you are building a valid manylinux2014 wheel and trying to repair it down to manylinux1. You can tell auditwheel what it should target with --plat.

Though if you are trying to build distributable wheels, I highly recommend trying cibuildwheel, that's what it's designed for.

henryiii avatar Jun 08 '22 01:06 henryiii

I was expecting auditwheel to "repair" it to the platform it deduced it was, instead of the default "linux_x86_64". And that is what I expect that "python -m build" would do too -- either by default or with some option. Looks like I'll be parsing the output of "auditwheel show" and renaming the wheel myself (maybe there's an auditwheel API to get the platform without the extra verbage?). Thanks for the tip about cibuildwheel. It seems like overkill, but I will take a look.

gregcouch avatar Jun 08 '22 23:06 gregcouch

Unless you are running inside a manylinux docker container, how would you deduce a platform? An arbitrary linux machine is not a safe place to build wheels; you should be running in manylinux (or musllinux). And if you are inside a manylinux docker image, you should know what docker image you are inside. (cibuildwheel or multibuild does all this automatically for you).

This is not something build does at all. Build makes a native wheel, not a redistibuitable one. There are places you need a native, non distributable wheel, like for CUDA, internal distribution on a homogeneous compute cloud, etc. auditwheel (plus the manylinux images) makes redistributable wheels on Linux from the native one. Delocate (plus the official macOS downloads and a few settings) makes redistributable wheels on macOS from the native one. Windows is pretty easy, any build of Python works, but you still might need develwheel. Again, cibuildwheel does this for you. If you don't want to use it, you need to learn how it works to implement it yourself (~100-500 LoC on CI, from when I've done it before).

henryiii avatar Jun 09 '22 00:06 henryiii

(This does depend on what you are using - if you are not using GLIBC, like with Rust, it becomes much easier, and the docker image isn't as important)

henryiii avatar Jun 09 '22 00:06 henryiii

In this case, I am using a CentOS 7, aka manylinux2014, (singularity) container. So hard-coding it would work. But I'm trying to make the build process generic to understand more about Python packaging is currently supposed to work.

In my main work, we use Python inside an application. And we use wheels internally to do upgrades between releases. There the Linux platform name doesn't really matter because it is all controlled by the application. But at some point in the future, we would like to put the independently useful packages on pypi with the correct platform tags. And it would be nice to just give an option to build that gives the wheel the redistributable name. SInce cibuildwheel is today's answer to the problem, I'll take a look.

gregcouch avatar Jun 09 '22 01:06 gregcouch