packaging.python.org icon indicating copy to clipboard operation
packaging.python.org copied to clipboard

Document real compatibility tag interpretation

Open lordmauve opened this issue 1 year ago • 0 comments

Issue Description

The compatibility tag section of the documentation is lifted directly from PEP-425 and covers how compatibility tags are constructed.

However, it is incomplete and inaccurate regarding how those tags are matched by pip. The described algorithm is:

This example list is for an installer running under CPython 3.3 on a linux_x86_64 system. It is in order from most-preferred (a distribution with a compiled extension module, built for the current version of Python) to least-preferred (a pure-Python distribution built with an older version of Python):

  1. cp33-cp33m-linux_x86_64
  2. cp33-abi3-linux_x86_64
  3. cp3-abi3-linux_x86_64
  4. cp33-none-linux_x86_64*
  5. cp3-none-linux_x86_64*
  6. py33-none-linux_x86_64*
  7. py3-none-linux_x86_64*
  8. cp33-none-any
  9. cp3-none-any
  10. py33-none-any
  11. py3-none-any
  12. py32-none-any
  13. py31-none-any
  14. py30-none-any

This is modified by "Compressed Tag Sets" but that's not relevant to this issue.

From this we can surmise, for a CPython version X.Y:

  • It can match if python is cpXY and abi is cpXY*.
  • It can match if python is in cpXY, cpX and abi is abi3
  • It can match if python is in cpXY, cpX, pyXY, pyX and abi is none
  • It can also match any pyXZ-none-any for Z < Y

This is not the actual algorithm used by pip.

In particular I'm aware of these rules obeyed by pip in practice (but I haven't found a spec):

  • It can also match any cpXZ-abi3 for Z < Y
  • It can also match any pyXZ-none-manylinux* for Z < Y

Given these are valid, they must be in the ordering somewhere, and it should be documented where.

Some examples illustrating these rules (pip 24.3.1, python 3.13):

$ .venv/bin/pip --version
pip 24.3.1 from /home/mauve/dev/demo/.venv/lib/python3.13/site-packages/pip (python 3.13)
$ .venv/bin/pip install --dry-run solace_pubsubplus --no-deps --no-cache
Looking in indexes: https://artifactory/pypi/simple/
Collecting solace_pubsubplus
  Downloading https://artifactory/pypi/packages/packages/5e/33/e5923f962b89b5a63eac2c0f7493abb53e7a1054a1da580c0037fd51713f/solace_pubsubplus-1.9.0-py36-none-manylinux_2_12_x86_64.whl (4.4 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 4.4/4.4 MB 549.6 MB/s eta 0:00:00
Would install solace-pubsubplus-1.9.0
$ .venv/bin/pip install --dry-run argon2_cffi_bindings --no-deps --no-cache
Looking in indexes: https://artifactory/pypi/simple/
Collecting argon2_cffi_bindings
  Downloading https://artifactory/pypi/packages/packages/ec/f7/378254e6dd7ae6f31fe40c8649eea7d4832a42243acaf0f1fff9083b2bed/argon2_cffi_bindings-21.2.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (86 kB)
Would install argon2-cffi-bindings-21.2.0

Given also that Compressed Tag Sets modify the whole algorithm, making it about set intersection, it would also be better to incorporate them into a description of the algorithm too.

Code of Conduct

  • [X] I am aware that participants in this repository must follow the PSF Code of Conduct.

lordmauve avatar Nov 21 '24 10:11 lordmauve