warehouse icon indicating copy to clipboard operation
warehouse copied to clipboard

Ban macOS 11+ minor tags not supported by pip, packaging nor uv

Open konstin opened this issue 1 month ago • 6 comments

See https://github.com/pypa/packaging.python.org/issues/1933

When macOS switched to a versioning scheme incrementing the major version with macOS 11, packaging and subsequently pip and uv decided to only emit tags with the minor version 0 for macOS 11+ (since we don't know how many minors each version has). PyPI doesn't currently enforce this requirement, to the confusion of users who upload wheels that are ultimately not usuable (https://github.com/astral-sh/uv/issues/16337). This PR bans uploads of those wheels, enforcing a minor version of 0 for macOS 11+.

Matching implementations in pip and uv:

https://github.com/pypa/packaging/blob/a85e63daecba56bbb829492624a844d306053504/src/packaging/tags.py#L452-L460

https://github.com/astral-sh/uv/blob/64bcd4e8a6ad629fa9ffc2922b4cc0cd57cb8dbe/crates/uv-platform-tags/src/tags.rs#L528-L538

DPO thread: https://discuss.python.org/t/codify-that-macos-platform-tags-use-minor-version-0-for-macos-11/104616

konstin avatar Oct 29 '25 10:10 konstin

Thanks for the PR!

Seems like this is the right thing to do, but I would like to see this made explicit in the spec at https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#macos before merging.

Do we know where wheels with invalid platform tags are coming from? I.e. is there some build tool producing these that we should try to fix first?

This appears to happen infrequently enough that we can probably do this without needing a deprecation period:

warehouse=> WITH versions AS (
  SELECT
    (regexp_matches(filename, 'macosx_(\d+)_(\d+)'))[1]::integer AS major_version,
    (regexp_matches(filename, 'macosx_(\d+)_(\d+)'))[2]::integer AS minor_version,
    upload_time
  FROM
    release_files
  WHERE
    filename ~ 'macosx_(\d+)_(\d+)'
)
SELECT
  DATE_TRUNC('month', upload_time) AS upload_month,
  COUNT(*) AS file_count
FROM
  versions
WHERE
  major_version > 11
  AND minor_version != 0
GROUP BY
  upload_month
ORDER BY
  upload_month DESC;

    upload_month     | file_count
---------------------+------------
 2025-10-01 00:00:00 |         12
 2025-09-01 00:00:00 |          1
 2025-06-01 00:00:00 |         12
 2025-04-01 00:00:00 |         14
 2025-03-01 00:00:00 |          8
 2025-01-01 00:00:00 |          2
 2024-12-01 00:00:00 |          7
 2024-11-01 00:00:00 |         10
 2024-10-01 00:00:00 |         49
 2024-09-01 00:00:00 |         30
 2024-08-01 00:00:00 |        101
 2024-07-01 00:00:00 |         33
 2024-06-01 00:00:00 |          6
 2024-05-01 00:00:00 |          2
 2024-04-01 00:00:00 |          9
 2024-03-01 00:00:00 |         10
 2024-02-01 00:00:00 |         15
 2024-01-01 00:00:00 |         20
 2023-12-01 00:00:00 |         30
 2023-11-01 00:00:00 |         32
 2023-10-01 00:00:00 |         38
 2023-09-01 00:00:00 |         36
 2023-08-01 00:00:00 |         31
 2023-07-01 00:00:00 |         30
 2023-06-01 00:00:00 |          2
 2023-05-01 00:00:00 |          6
 2022-07-01 00:00:00 |          3
 2022-02-01 00:00:00 |          2
(28 rows)

di avatar Oct 29 '25 13:10 di

Thanks for checking the DB!

I've updated the description with the matching DPO thread: https://discuss.python.org/t/codify-that-macos-platform-tags-use-minor-version-0-for-macos-11/104616. I've been told we need a DPO thread to change packaging.python.org, even though there's already enforcement through the packaging implementation (and uv following that), it's a convoluted situation.

Do we know where wheels with invalid platform tags are coming from? I.e. is there some build tool producing these that we should try to fix first?

No idea, the one report I have from uv is closed source. Setuptools has the correct behavior: https://github.com/pypa/setuptools/blob/8653b91b2b3bdf8a6b77a26d0aa4dd28ffda9bc5/setuptools/_vendor/wheel/macosx_libfile.py#L420-L421

konstin avatar Oct 29 '25 15:10 konstin

No idea, the one report I have from uv is closed source.

For at least one recent wheel, the wheel generator is claimed to be uv itself: https://inspector.pypi.io/project/pyvoy/0.0.1/packages/fa/5a/abfc49c65fac6f839cd25d76da1f572f60f14c4fa826906a2bd4a520d53d/pyvoy-0.0.1-cp310-none-macosx_14_5_arm64.whl/pyvoy-0.0.1.dist-info/WHEEL#line.2

Looks like for another it's hatchling (cc @ofek): https://inspector.pypi.io/project/mvsr/0.1.2/packages/55/f7/538a3f40d8f0e2d62455a2a88b5a3a6a487c14c6755a4830d8b05bdf9d08/mvsr-0.1.2-py3-none-macosx_15_6_arm64.whl/mvsr-0.1.2.dist-info/WHEEL#line.2

di avatar Oct 29 '25 16:10 di

https://github.com/Loesgar/mvsr/blob/main/lang/python/hatch_build.py

ofek avatar Oct 29 '25 17:10 ofek

For at least one recent wheel, the wheel generator is claimed to be uv itself: inspector.pypi.io/project/pyvoy/0.0.1/packages/fa/5a/abfc49c65fac6f839cd25d76da1f572f60f14c4fa826906a2bd4a520d53d/pyvoy-0.0.1-cp310-none-macosx_14_5_arm64.whl/pyvoy-0.0.1.dist-info/WHEEL#line.2

Huh, the uv build backend does write py3-none-any wheels exclusively, this must be manual editing or another claiming to be uv.

konstin avatar Oct 29 '25 17:10 konstin

Thanks, both of those confirm my suspicion that this is mostly due to users manually meddling with the tags.

di avatar Oct 29 '25 18:10 di

There were no negative comments on the DPO thread and it's been a while, so I opened a packaging.python.org PR: https://github.com/pypa/packaging.python.org/pull/1958

konstin avatar Nov 21 '25 10:11 konstin

Just noticed this thread - sorry it looks like I'm missing a tag to link the GitHub repo on pypi, the source for creating the wheel is here, using the wheel command.

https://github.com/curioswitch/pyvoy/blob/15a3451f2d222b25913eb9540e91df2735f3de34/scripts/wheel.py#L37

I found it the easiest way to create native wheels that aren't Python extensions.

So I guess the wheel command should also be updated to validate the Mac platform tags? Though it looks like that command doesn't really do any validation.

anuraaga avatar Nov 23 '25 01:11 anuraaga

That's a question for https://github.com/pypa/wheel/ more than for the PyPI.

konstin avatar Nov 24 '25 08:11 konstin

The packaging.python.org PR merged (https://github.com/pypa/packaging.python.org/pull/1958)

konstin avatar Nov 24 '25 08:11 konstin