rules_py icon indicating copy to clipboard operation
rules_py copied to clipboard

[FR]: Support multiple Python toolchains

Open JaredNeil opened this issue 1 year ago • 3 comments

What is the current behavior?

rules_py doesn't seem to have an easy way to use multiple python toolchains in the same repository.

Describe the feature

When using python_register_multi_toolchains from bazelbuild/rules_python, it creates py_test and py_binary macros for each version of python that are loaded from @python//<version>:defs.bzl. It would be nice to have a rules_py analog to python_register_multi_toolchains, or some easy way to have rules_py's py_* rules use multiple python toolchains in a single workspace.

JaredNeil avatar Mar 27 '23 23:03 JaredNeil

This works for me using bazel modules:

python = use_extension("@rules_python//python/extensions:python.bzl", "python")
python.toolchain(
    is_default = True,
    python_version = "3.10",
)
python.toolchain(
    python_version = "3.11",
)
python.toolchain(
    python_version = "3.12",
)
use_repo(
    python,
    "python_3_10",
    "python_3_11",
    "python_3_12",
    "python_versions",
)

Then edit in your flags as desired to the .bazelrc:

build:310 --@rules_python//python/config_settings:python_version="3.10"
build:311 --@rules_python//python/config_settings:python_version="3.11"
build:312 --@rules_python//python/config_settings:python_version="3.12"

When executing a target such as:

load("@aspect_rules_py//py:defs.bzl", "py_binary")

py_binary(
    name = "main",
    srcs = ["main.py"],
)

finn-ball avatar Apr 11 '24 13:04 finn-ball

Would that let you define a 3.10 py_test and a 3.12 py_test in a the same package and run them both in the same bazel test invocation, like the example linked in the first comment?

JaredNeil avatar Apr 11 '24 15:04 JaredNeil

The example linked uses the rules py_test_3_8, py_test_3_10 etc as separate targets.

The way I've just sent allows you to register multiple toolchains and then launch multiple invocations of bazel but with different toolchains on the same target.

e.g

bazel test //... --config=310
bazel test //... --config=311

finn-ball avatar Apr 11 '24 15:04 finn-ball