Override a requirement with a Bazel target

[dhalperi]

🚀 feature request

Relevant Rules

load("@rules_python//python:pip.bzl", "pip_install")

Description

We'd like to be able to override a requirement with a Bazel target. This is basically the rules_python equivalent of rules_jvm_external's override_targets directive.

Consider the case where we're using grpc and protobuf from Bazel, and an external python library depends on protobuf==3.13.0 (or whatever). I'd like to replace that dependency with "@com_google_protobuf//:protobuf_python" instead, so I'm not mixing pypi and Bazel versions of protobuf.

Describe the solution you'd like

I like the way that rules_jvm_external has done it.

Describe alternatives you've considered

ø

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

[psigen]

Bumping because I don't want this to be closed without anyone seeing it, it seems like a good idea :sweat: Hopefully this keeps the stalebot away for a bit.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

[dhalperi]

Bumping because I don't want this to be closed without anyone seeing it, it seems like a good idea 😓 Hopefully this keeps the stalebot away for a bit.

[philsc]

I pushed a branch to my fork here: https://github.com/philsc/rules_python/compare/add-deps-injection...philsc:rules_python:add-target-override that should add this feature.

I'm proposing that this be done via an annotation: https://github.com/philsc/rules_python/compare/add-deps-injection...philsc:rules_python:add-target-override#diff-34dce464ac9dc63ca9a3f06b5950d01f57bf6a77fe9c5d39ed3160cec9c05628R46-R50

I can put it up as a PR once #853 is merged.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

[dhalperi]

Bumping because I don't want this to be closed without anyone seeing it, it seems like a good idea 😓 Hopefully this keeps the stalebot away for a bit.

[arrdem]

This seems like a special case of the general gap between rules_go and rules_python wrt the patches= directive enabled by generating a static repository list.

[alexeagle]

I think this is handled in https://github.com/aspect-build/rules_py with upcoming work @mattem is open-sourcing that allows "virtual" dependencies, where the version of some import can be overridden by an application rather than always using the one provided transitively by a library target the application depends on.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had any activity for 180 days. It will be closed if no further activity occurs in 30 days. Collaborators can add an assignee to keep this open indefinitely. Thanks for your contributions to rules_python!

[arrdem]

We've wound up having to implement something like this. tensorflow has ah fragmented packaging, so we've wound up having to build a bunch of machinery so that py_library can recognize pseudo-labels generated by py_requirement and rewrite them into an appropriately chosen real rule label.

tensorflow is an especially nasty case of this because the name of the correct package to choose is contingent on the target platform and architecture, but since rules_python can only materialize 3rdparty repos for architectures matching the build host that means the correct choice of tensorflow is defined by host-specific workspace configuration which we already have to choose the "right" requirements.txt lock for the current platform.

Significantly we can't use select() to implement the same behavior by say creating an alias rule across underlying tensorflow labels because in most build configurations most of the arms of the select would point to lables in external workspaces that can't be fetched/materialized and thus break lots of different analysis usage.