uv
uv copied to clipboard
astral-sh
`uv pip install` fails for dependencies on already-installed local packages
When a package depends on an editable-installed namespaced package, uv pip install fails saying that the already-installed package can't be found. Possibly related to string hygiene and converting ns1.package1 to ns1-package1?
Full cloneable repro case here: https://github.com/charlesnicholson/uv-pep420-bug/tree/main (Runs at least on macOS and maybe Linux, let me know if it's insufficient and I'll fix it up)
Confirmed still happens on 0.1.4 (albeit with a much more soothing color scheme!):
So, I think cargo run pip install -e ns2.package2 -e ns1.package1 works, but installing the packages one-by-one does not. I need to think about what the right solution is here. When we go to install the second package (-e ns2.package2), we start by performing a resolution, to figure out the required dependencies. We don't consider the "current virtual environment" to be a valid package source, so we have no way of finding ns1-package1.
Ah, interesting, thanks for taking a look and sharing your thoughts!
For context, if it's useful, the reason we incrementally pip install -e our various packages in order is because they're part of a larger build system that understands the dependencies between our packages and other tools. Each package has an "editable install" target, and for our build system to satisfy our version of ns2.package2's dependencies, it needs to bring the "editable-install ns1.package1" target up-to-date first, which it does via a separate invocation of pip install -e.
OTOH maybe since uv is so hilariously fast, there isn't really a build-time performance penalty for just always re-editable-installing our dependent packages along with the new package :-D (Transitive package dependencies make this complicated though)
Doing the same work with pip definitely incurs a performance penalty.
I am curious, though- the venv seems like a pretty reasonable source of packages given that it holds strongly-versioned already-installed dependencies, and that's more or less its reason to exist. To my mind, it seems reasonable that uv would behave identically with both a unified and multiple separate invocations. I am a brand new user, though, and not the author, so I certainly don't have the context that you do.
I think it's probably correct for us to fix this. It just changes a lot of assumptions -- right now, we perform a resolution that's independent of the virtual environment.
Yeah, the problem is that the second install doesn't "know" where to find the existing editable. We need to make the resolver aware of packages that already exist in the virtual environment.
Possibly related (if not, I can file a separate issue—or none at all if this is a misunderstanding on my part), when I uv pip compile a requirements.in or uv pip install a requirements.txt that has local, editable-installed modules (e.g, specified with -e path/to/module) they are all rebuilt and I assume cached, even if they haven't changed and don't need it. Is this avoidable?
For the record, this is a large monorepo with dozens of modules. The commands complete really fast and therefore it's not a huge problem, just wondering if there is a way (or possibly a different work flow) to prevent this.
@tylerw -- I think we try to avoid rebuilding with uv pip compile, but we always rebuild right now in uv pip install. Building editables is typically pretty cheap, but I'm looking into removing that step.
Hi! Charlie tricked me into taking on this hard task instead :)
It should be addressed in the latest release (0.1.27) via #2596.
Let us know if you have any problems.
So far it's great, thanks! (I'm integrating it as I type this :) )