rules_pycross icon indicating copy to clipboard operation
rules_pycross copied to clipboard

Support uv.lock extras

Open keith opened this issue 10 months ago • 5 comments

For some complex configurations uv recommends using extras to install different versions of the same dependency:

https://github.com/astral-sh/uv/blob/dc39d6622be268318acfb43a758c6ea688b722ea/docs/guides/integration/pytorch.md#configuring-accelerators-with-optional-dependencies

In this examples those extras markers are encoded in the uv.lock, and it appears that today rules_pycross takes 1 version, and ignores the potential for other versions.

I think in an ideal world the extras would be configurable through some string_flag (and therefore also a transition).

From a user POV I think the alternative is to avoid this feature but then have N uv.lock files and generate separate bzl files for each configuration, which could be a bit unwieldy (but also what I do with requirements.txt files today with rules_python)

keith avatar Feb 28 '25 20:02 keith

I can't tell if this is something rules_pycross already supports with poetry or not, I might try that to see if it's similar

keith avatar Feb 28 '25 21:02 keith

looks like poetry actually fails in that case since it uses the version info for this instead, and that is expected to be a type that it is not

keith avatar Feb 28 '25 21:02 keith

an alternative solution for this would be for me to somehow inject the label for the dep, that might be a transitive of other deps, so that I could handle the complexity of different versions behind my own select()

keith avatar Feb 28 '25 22:02 keith

Are these the same as traditional package "extras"? I'm having a difficult time determining from that doc.

If so, I've got a branch somewhere where I started adding native support for package extras, but there's still a ways to go.

jvolkman avatar Mar 02 '25 03:03 jvolkman

I think it's an entirely separate concept borrowing the same terminology. I think the goal of it is just to allow you to install different versions of things based on different user facing (or hardware) configurations with a single lockfile. So in the torch example your lockfile can contain a torch version for CPU or GPU, and then if you were to use uv to create a venv for you, you could pass --extras cpu to get 1 vs the other. So the extras are entirely defined by you and your lockfile.

That's why I was kind of thinking a string_flag or similar in bazel would be a good analogy, similar to defining the extras for the uv venv, it could control which of those deps was used in the tree, while defining bazel targets for all of them.

keith avatar Mar 03 '25 19:03 keith