rules_python

rules_python copied to clipboard

PR Checklist

Please check if your PR fulfills the following requirements:

• [ ] Tests for the changes have been added (for bug fixes / features)
• [ ] Docs have been added / updated (for bug fixes / features)

PR Type

What kind of change does this PR introduce?

• [ ] Bugfix
• [x] Feature (please, look at the "Scope of the project" section in the README.md file)
• [ ] Code style update (formatting, local variables)
• [ ] Refactoring (no functional changes, no api changes)
• [ ] Build related changes
• [ ] CI related changes
• [ ] Documentation content changes
• [ ] Other... Please describe:

What is the current behavior?

Running pip_parse or whl_library on Linux with a requirements.txt for Windows downloads the Linux instead of the Windows version of wheels.

Adding extra_pip_args = ["--platform", "win_amd64"] doesn't work because --platform isn't a valid flag for pip wheel.

In the case that all pip dependencies are available as precompiled wheels, pip download can be used instead, which accepts the --platform flag.

Issue Number: N/A

What is the new behavior?

Add a new option download_only to pip_parse and whl_library to allow using pip download instead of pip wheel so that wheels for platforms other than the host platform can be downloaded by adding --platform ... to extra_pip_args.

Does this PR introduce a breaking change?

• [ ] Yes
• [x] No

Other information

@groodt would you be able to take a look here some time this week? This change would be very helpful!

(also thanks for all the support!)

It might be useful to document how this can be used to target multiple platforms. Example:

# WORKSPACE

load("@rules_python//python:pip.bzl", "pip_parse")
load("@python3_10//:defs.bzl", "interpreter")

# Repeat for each supported target platform
pip_parse(
    name = "pip_windows",
    python_interpreter_target = interpreter,
    requirements_lock = "requirements_windows.txt",
    download_only = True,
    extra_pip_args = ["--platform", "win_amd64"],
)

# BUILD

load("@rules_python//python:pip.bzl", "compile_pip_requirements")

# Repeat for each supported target platform
compile_pip_requirements(
    name = "requirements_windows",
    requirements_in = "requirements.in",
    requirements_txt = "requirements_windows.txt",
    target_compatible_with = [
        "@platforms//os:windows",
        "@platforms//cpu:x86_64",
    ],
)

# Repeat for each supported target platform
config_setting(
    name = "windows_x86_64",
    constraint_values = [
        "@platforms//os:windows",
        "@platforms//cpu:x86_64",
    ],
)

# Repeat for each pip dependency
alias(
    name = "numpy",
    actual = select({
        # Repeat for each target platform
        ":windows_x86_64": "@pip_windows_numpy//:pkg",
        # ...
    }),
    visibility = ["//visibility:public"],
)

# ...

Obviously can be simplified with macros and loops when there are multiple target platforms and pip dependencies.

I’m a big fan of this change, or something similar! Out of curiosity, why this PR vs. https://github.com/bazelbuild/rules_python/pull/510?

This is definitely simpler! But #510 comes with the aliasing for platforms already?

@schultetwin I didn't know about #510 and I don't have a strong preference between either PR.

It would be nice if there was a properly supported cross-build workflow like #510 is aiming for but it has been open for a year and there appears to be some open design questions.

This PR is a simpler change to at least unblock the ability to do cross builds as detailed above in https://github.com/bazelbuild/rules_python/pull/773#issuecomment-1202118678.

This PR is a simpler change to at least unblock the ability to do cross builds as detailed above in https://github.com/bazelbuild/rules_python/pull/773#issuecomment-1202118678.

Understood, thank you!

Thank you for this change, it saved me

@jesseschalken @groodt

Is there a way to make this work with Gazelle? It would be awesome if gazelle can understand the select({...}) to work with multiple repos.