uv

uv copied to clipboard

Not sure if we want this -- it's like Poetry's --no-root. Let's wait and see if we can find motivating use-cases for it. Poetry also has this concept of "operating modes" which toggle this automatically.

(Not decided, filed to revisit later.)

Lots of prior chatter over in https://github.com/python-poetry/poetry/issues/800

Maybe see also https://github.com/pypa/pip/issues/11440 for potential use cases.

(Interestingly, we actually kind of already support what's described in that issue because we allow pip install -r pyproject.toml. But that's separate from this issue.)

(Interestingly, we actually kind of already support what's described in that issue because we allow pip install -r pyproject.toml. But that's separate from this issue.)

That doesn't support the [extras] syntax (e.g. pip install -r pyproject.toml[doc])?

Personally, I'd love to see support for the syntax suggested here:

pip install --only-deps  .[doc]  # project.dependencies and project.optional-dependencies.doc

[EDIT] Just discovered that pip install -r pyproject.toml --extra doc gives the functionality I was looking for. That's really great!

This works great pip install -r pyproject.toml :) It would be still great to add a flag and also add it to the pyproject.toml

[tool.uv]
package-mode = false

So it would also work with uv add <package>. Many things that I work on are not packages, but it would still be great to use uv to manage dependencies.

PS: uv is such an amazing tool - thanks for building it ❤️

I have a use case for that: my blog, which is actually Markdown content only, and is built by Pelican, a pure Python static website generator.

In that case the package-mode = false from Poetry was handy.

In the mean time, I hacked the process and added a dummy package at the root of my repository:

$ ls -lah ./blog
Permissions Size User Date Modified    Name
drwxr-xr-x     - kde  2024-07-13 14:12  .
drwxr-xr-x@    - kde  2024-07-13 14:17  ..
.rw-r--r--     0 kde  2024-07-13 14:11  __init__.py

And added the following directive in pyproject.toml:

[tool.setuptools.packages.find]
include = ["blog"]

My use case: ansible project with bundled ansible and other related packages.

For ansible project, the majority of code are written in yaml, by providing pyproject.toml we can build a self-contained ansible environment in project's .venv folder, with guaranteed ansible modules versions.

Besides, non-python-package programs like sshpass can be distributed into .venv/bin, so by source .venv/bin/activate ansible will be able to use functionally like remote host with password.

We use --no-root to better caching of container image layers.

The first layer just installs the project dependencies; the second adds the projects itself. So if dependencies don't change, the layer can be reused.

We hope to write something like:

WORKDIR /app
ADD pyproject.toml uv.lock /app
RUN uv sync --no-dev --no-root
ADD src /app
RUN uv sync --no-dev

In the above scenario of a layered Dockerfile to leverage caching of infrequently changing dependencies, it is important that there is a way to install from uv.lock without involving pyproject.toml. That is because when pyproject.toml gets involved, it may in turn require the presence of the whole project (because of an in-tree build backend, dynamic dependencies, etc).

At first sight, uv.lock contains everything that is needed to do this, except for an indication about the top level package(s).

IMHO this is a critical feature for adoption of uv.lock, as without it the space and time efficiency of dockerfile builds will be severely impacted, compared to the traditional approach of installing requirements.txt in a layer, then the app in another.

I agree that something like this is necessary.

I'll chime in that --no-root is not enough. You probably also want a --no-directory or --no-local, maybe just generalize to a source selector or something.

Motivation: https://python-poetry.org/docs/faq#poetry-busts-my-docker-cache-because-it-requires-me-to-copy-my-source-files-in-before-installing-3rd-party-dependencies

In particular the idea is that if you have a monorepo you can still copy in your top-level lockfile and not have to copy in anything (not even the pyproject.toml) from your sub-projects/packages.

Yeah 100% agree that you should be able to sync from just the lockfile. The lockfile itself is actually designed to support that (it doesn’t rely on pyproject.toml), but we need to expose the right options to users…

I can see a few approaches here (all fairly easy to implement, mostly about getting the API right)...

• --no-project (or --no-workspace) along with a separate --no-local flag (to ignore path dependencies that aren't part of the workspace)
• --only-remote (the inverse)
• Something like --seed or another semantic term that indicates why this is useful

In case you’re looking for opinions, I like “seed”. The others are rather generic and I’d have to guess what they exactly mean.

(Gonna see if we can come up with an API and ship this ~today.)

Opinions very welcome.

I think --no-project is pretty clear but don't care too much as well as it's documented.

Would --no-self and --only-self be more intuitive?

--seed {package-name}[extra,...], because you need to tell it what top level package's dependencies you want to install?

Some thoughts....

I think extras would be covered by the normal --extra and --all-extras options.

I worry about --no-project and --no-workspace because those have a different meaning elsewhere; which is, don't discover a project or workspace entirely. We still should perform normal project or workspace discovery, e.g., to find the uv.lock file if you're in a subdirectory for some reason.

I think we might want to use --without- and --only- prefixes, to match the our --with options.

The difficulty is in choosing how to describe what we're selecting. There are a few things we want to be able exclude:

• The current project
• A workspace member
• A path dependency

Unfortunately --without-editables doesn't work because a path dependency can be either editable or not editable. We also want to be able to toggle whether or not the current project and workspace members are installed as editable in the future.

All of these are "local" dependencies, as in they're on the local file system, so --without-local might make sense. Does this include the current project though? Do we need an inverse flag, like --only-remote? Is that easier to reason about?

It feels like we need a separate flag for excluding just the current project, like --without-self or --without-root (I think --without-project isn't an option because it overlaps too much with --no-project). Presumably we'd pair this with an inverse flag like --only-self or --only-root.

Do we need the ability to exclude all local dependencies except the current project? Or can it be done in two steps:

uv sync --without-local
uv sync --inexact --only-self

What's the use-case for that?

I think the only use-case for the Docker caching situation is "exclude all locals".

I think excluding the current project (but not other workspace members) is independently useful, like for "virtual projects." We could choose not to solve that here, but we should probably have a design that won't cause us problems in the future if we pursue it.

As a note on my own post, I think I prefer --without-locals over --without-local?

What about --no-locals or --no-local to match options like --no-project?

Ok, I didn't get this shipped today, but I did implement it: https://github.com/astral-sh/uv/pull/6398

The gist is that you can copy in uv.lock, then run uv sync --frozen --no-locals.

Well, I specifically opted for --without- instead of --no- since we use --with- for including dependencies it probably makes sense to do the same for excluding them?

I would be remiss if I didn't mention that this feature would be massively more useful (and less surprising, I guess) in the Docker context if uv sync --python /some/venv/bin/python worked as expected. 😇

But mega kudos for the quick implementation as always; this is amazing! 🚀

We'll definitely be tackling alternative sync targets ASAP as well.

I think the docker case is real but at the end of the day it creates a situation in the virtualenv that I would argue should not exist. So at least in my mind there is a significant difference between "install third party deps", make a layer, then follow by installing the rest vs "never ever install the root package because it's inconvenient".

Particularly for the docker situation I quite honestly wonder if not a fundamentally better solution is to allow the installation process to be explicitly being built for that use case by supporting skipping/interrupting and continuing. I understand that this is not how any other tool works but particularly for more complicated setups I can see this being quite useful.

I'm imagining that this might be quite interesting. This would work if the lockfile is up to date prior to the sync.

# first layer, third party deps only
RUN uv sync --partial --skip-package="root-package" --skip-package="second-package"
COPY src/root-package ./src/root-package
# second layer, bring one package in
RUN uv sync --partial --skip-package="second-package"
COPY src/second-package ./src/second-package
# sync what we have not seen yet
RUN uv sync

At sentry for instance we do install packages from two different repositories (open source git checkout followed by internal billing code checkout).

I think being more explicit about what this is used for would be a good thing and also forces that the docker case is intentionally supported on the uv side and not just a happy little accident.

This also leaves the option open to yell at the user if the virtualenv is being utilized and does not actually reflect a full sync.