WIP: ENH: implement support for build-details.json (PEP 739) and cross compilation
This is very much not ready yet, but I'm opening a WIP PR now to get early feedback, since there's more than one way this can be implemented.
Depends on https://github.com/mesonbuild/meson/pull/14657 (Cc @mgorny) Closes gh-321
Example of a cross compilation CI job for SciPy, which successfully uses a cross file and a build-details.json file to cross compile from x86-64 to aarch64 and then in a second job installs and tests the cross-compiled wheel:
- https://github.com/scipy/scipy/compare/main...rgommers:scipy:ci-cross
- https://github.com/rgommers/scipy/actions/runs/16294476137
The meson-python parts of cross compilation that need fixing are:
- The wheel filename (interpreter, abi and platform tags)
- The platform tag in the
WHEELmetadata file
The changes here do both of those things.
A few things I'd like to get input on:
- I chose to use both
is_crossandbuild_detailsin_WheelBuilder, for clarity and so that cross compilation without use of abuild-details.jsonfile can also be supported (parsing the cross file can get the platform right at least, but not the interpreter). We could also decide to not worry about that case and only support supplying both files for a cross build. - Test strategy:
- Cross compile a single package, or multiple/all?
- Do we do just Linux x86-64 -> aarch64 and vice versa as tests where we can validate at runtime because easy with GHA runners and toolchain setup, or anything else?
- Conda-forge is easiest to set up (for me at least), but we can also do another distro.
- Use minimum supported Python and Python 3.14? For the former we need to generate a
build-details.json, for the latter it's shipped as part of CPython.
- (minor) Static typing:
build_detailsis hard to type. Usebuild_details: Optional[Dict[str, str | Dict[str, Any]]]or simplyDict[str, Any]?
I haven't read PEP 739 recently, but I was under the impression that PEP 739 will also be used for building native wheels, not only for cross compilation, or in other words that given a Python interpreter it should be possible to load a build-details.json describing the interpreter, sparing build tools from doing the kind of messy interpreter introspection meson-python needs to implement in _tag.py. However, as implemented here, the support for PEP 739 is enabled only in the case of cross compilation. Did I misunderstand the intent of PEP 739?
Good point. Yes, there isn't any reason that build-details.json cannot be used for a native build. So we should support that as well. That said, there isn't a benefit from the user passing in such a file explicitly; we can start reading it automatically for Python >=3.14, but I'm a bit hesitant to do that right away since it's so new. We can start doing that at any point in the future if it helps simplify our internal code, since it's a backwards-compatible change.
So for now in this PR we can distinguish three cases, with the user passing:
- Only a cross file (or multiple cross files)
- Only a
build-details.jsonfile - Both a cross file and a
build-details.jsonfile
I focused on (3) first and have a TODO in here already to add (1); I should add (2) as well.
Note that https://github.com/mesonbuild/meson/pull/14657 have been merged and is available since version 1.10.0