[FR]: py_pex_binary support for arbitrary cross-builds

[plobsing]

What is the current behavior?

Due to the slightly hacky way py_pex_binary disassembles the output of a py_binary and reconstitutes it into an archive, it pulls in certain components as dependencies that don't make it into the final archive. But because they are tracked as dependencies, they need to be buildable for the target platform anyways or the build fails.

Notable among these is //py/tools/venv_bin:venv, which is a runtime dep for py_binary, but is not useful in the context of PEX. But, owing to its implementation either as a pre-built binary (in released rulesets) or a Rust binary (in dev rulesets), the targets that it is available for are limited. And this limitation is viral — it also applies to the overall PEX archive that the binary is a dep of (but gets stripped out from).

For example, attempting to compile a PEX archive targetting MacOS from a Linux build host fails for the dev ruleset because the Rust toolchain does not support cross-compiling this way.

(04:17:00) ERROR: /mnt/ephemeral/workdir/aspect-build/rules_py/py/tools/venv_bin/BUILD.bazel:4:12: While resolving toolchains for target //py/tools/venv_bin:venv_macos_aarch64_build (d30093f): No matching toolchains found for types @@rules_rust//rust:toolchain_type.
To debug, rerun with --toolchain_resolution_debug='@@rules_rust//rust:toolchain_type'
If platforms or toolchains are a new concept for you, we'd encourage reading https://bazel.build/concepts/platforms-intro.

In release, this cross-compile would work because the prebuilt toolchain for MacOS exists. But we also see similar problems targetting Windows, or more exotic OSes.

Describe the feature

Cross-compilation targets for PEX archives should be not be limited artificially by rules_py; if the PEX tool can run on the exec platform, and the Python dependencies of the backing py_binary are compatible with the target platform[^1], we should be able to build an archive.

[^1]: Either because each dep is pure-python, or has a pre-built wheel that's been pulled from PyPI, or the user has provided some other builds that works for the target platform.

[plobsing]

FYI, I believe https://github.com/aspect-build/rules_py/issues/587 may improve the situation with the runtime dependency on the //py/tools/venv_bin:venv tool. I'm not sure if that gets this feature all the way across the line or not, but it just might.

[arrdem]

PEX, out of the box, provides crossbuild support by itself interacting with setuptools etc to fetch prebuilds compatible with a given target configuration. This doesn't fit with the Bazel best practices which are that downloads should strictly occur during the repository/module phase, and crosscompilation or cross-target selection of artifacts should occur in build configuration.

As you note the current implementation of py_pex_binary reconstitutes wheels which have been downloaded by the rules_python pip machinery and emplaced as installed Bazel libraries. This has long been the only available Python package manager implementation for Bazel, and the only transparent way to get wheel files from normal py_library targets as rules_python does not offer a build configuration flag allowing 3rdparty libraries to be transitioned to wheels.

As such the options here are:

1. Status quo; rules_python's pip doesn't support crossbuilds and so we can't either in any sort of principled way
2. Add special spooky machinery to py_pex_binary which enables us to drive pex to perform cache inefficient and nonhermetic downloads at action phase which is what you're asking for implicitly here
3. #650 which just makes crossbuilds work for generalized Python artifacts allowing our pex implementation to remain a fairly simplistic zipper

[plobsing]

This FR is not about doing the package management. I do not view package management as in-scope for py_pex_binary. I like that the archive builder relies on other parts of the Bazel/Python ecosystem for package management. I also like that it operates hermetically.

What is requested is that the archive building machinery not interfere with cross-builds that would otherwise work. I.e. assume I have all of my py_library and wheel inputs built for the target architecture ~somehow, the toolchain registrations and/or other platform constraints of the py_pex_binary and py_binary rules should not interfere with building the archive.

rules_python's pip package manager is not the only game in town when it comes to Python pacakge management. rules_pycross has existed for some time now and it supports cross-builds of third-party Python dependencies. This is how I have been obtaining my cross-built dependencies, and it has been working well for me in other contexts. And based on the debugging I did to try to get PEX archives to work, it seems to me I would be able to use my cross-built deps to form PEX archives if not for the spurious target-platform dependencies on binaries from @rules_py//py/tools/venv_bin (these toolchain binaries aren't available on all platforms, which is fine, they should only be needed on build machines).

[arrdem]

FYI arbitrary crossbuilds for py_venv_binary are now possible and working in #650, the other binaries should Just Work (TM) except where we have toolchain bugs such as you're describing. Those should be easy to fix.

It's also possible to transition() UV provided deps from being installs to just being whls so that wheel rezipping is no longer required.

There's about to be a much stronger game in town ;)

[arrdem]

@plobsing would you be able to share a representative requirements lock or such for the crossbuilds you're trying to do? That'd let me test the new UV machinery's ability to kick out windows artifacts.

[plobsing]

Here's a small, self-contained demo: https://github.com/plobsing/bazel_pex_cross_demo

Its great to hear that the ecosystem is getting new and better options. However, last time I checked, Uv was unable to find a solve for my real project's lockfile. I believe this was due to the use of mutually-exclusive extas listed in the pyproject.toml. I do not anticipate I will be able to use Uv soon or possibly even ever; I will likely continue to use PDM. The loose coupling of the Bazel Python environment makes this possible, and it would be very nice if that could continue (i.e. please don't couple the cross-build functionality too closely to Uv).