uv icon indicating copy to clipboard operation
uv copied to clipboard

Add `--no-install-*` to `add`/`remove`/`run` and add env vars

Open mkniewallner opened this issue 1 year ago • 15 comments
trafficstars

Some options were added in https://github.com/astral-sh/uv/issues/4028 to avoid the installation of some packages, including the project itself. Taking --no-install-project as an example, it works great when running uv sync --no-install-project, but as soon as we want to run a script/entrypoint through uv run later on, we'll still end up installing the project at some point.

For instance, if we want to build documentation for a project, running uv run mkdocs build will install the project itself, which could take some time if the project is using Rust.

Current workaround I found is to use uv run --no-project mkdocs build to avoid project discovery (assuming that uv sync --no-install-project was run beforehand), but I wonder if the 3 options added to uv sync should not also:

  • be added to other commands that call do_sync (add, remove, run)
  • be configurable via environment variables (this could be really useful on CI workflows)

mkniewallner avatar Aug 24 '24 11:08 mkniewallner

Can you share more about the use-case here? We added --no-install for Docker image building and layer caching. It's not intended for you to maintain a partial environment long-term.

I'm not necessarily opposed, but I think we should be careful here — we moved fast to deliver that feature because it's critical for Docker workflows but I'm not sure I understand this use-case as much.

zanieb avatar Aug 26 '24 13:08 zanieb

I also need this feature. Our use case is that we are frequently not creating python packages like this pattern assumes, but we are instead defining the dependencies using UV that are packaged alongside our source code which gets deployed into lambdas as part of serverless infrastructure. The .egg-info file that gets created for the project is unnecessary, especially since the various lambdas that compose the service will each independently define their own entry points (a single "handler" function) within the same src folder.

I'd also prefer to have the ability to set no_install_project = true in the pyproject.toml so that the project itself is never added regardless of uv command.

jfgordon2 avatar Aug 26 '24 15:08 jfgordon2

I don't think you're looking for --no-install-project. I mean, it might work for that use-case, but that isn't why it exists. You want something like #6585: the ability to mark projects as non-packages.

charliermarsh avatar Aug 26 '24 16:08 charliermarsh

Can you share more about the use-case here? We added --no-install for Docker image building and layer caching. It's not intended for you to maintain a partial environment long-term.

I'm not necessarily opposed, but I think we should be careful here — we moved fast to deliver that feature because it's critical for Docker workflows but I'm not sure I understand this use-case as much.

Yep agree with not providing too many options unless they really make sense.

My use case is for https://github.com/fpgmaas/deptry, where we use maturin to build a project with a mix of Python and Rust. Building deptry itself takes quite some time, since we need to compile Rust.

In the CI, we have a dedicated job that validates documentation with mkdocs, that we set as a dev dependency. The job only needs mkdocs, but dependencies installation is so fast with uv that we don't mind installing other dependencies as well (like mypy for instance).

But when we run uv run mkdocs, we end up installing deptry itself, which takes quite some time, although we don't need to do that in this specific case.

We technically have other options, so it's not a blocker at all. For instance I guess we could:

  • move uv run mkdocs to another job that does require to install deptry itself
  • set the documentation specific dependencies in docs/requirements.in and use uv pip compile
  • use workspaces?
  • use the --no-project workaround that we currently use for now
  • use uvx [email protected] (but we'd loose the ability to lock transitive dependencies)
  • ...or other things I might be missing?

Of course, since deptry is a library, we cannot use virtual projects in this specific case.

mkniewallner avatar Aug 26 '24 17:08 mkniewallner

Thanks for the details! We have the same problem here in the uv repository. We do

uvx --with-requirements docs/requirements.txt -- mkdocs serve -f mkdocs.public.yml

right now. Obviously this isn't ideal though. I think "project-level" tools is what you're looking for here (#3560) but we don't support that yet and need to do some design work first. I think --no-project is the "correct" workaround for now. This use-case is important, but I think uv run --no-install-* is probably not be a good long-term solution for it.

zanieb avatar Aug 26 '24 17:08 zanieb

I don't think you're looking for --no-install-project. I mean, it might work for that use-case, but that isn't why it exists. You want something like #6585: the ability to mark projects as non-packages.

Ah! I think I'm just making some assumptions based on my unfamiliarity with the terminology in this project so far - thank you for pointing out that PR! I'll just patiently wait for that one.

jfgordon2 avatar Aug 27 '24 02:08 jfgordon2

No prob, I mostly just think we can do a better job of solving your problem that way :)

charliermarsh avatar Aug 27 '24 02:08 charliermarsh

@jfgordon2 the alternate approach is available in uv 0.4.0 now.

zanieb avatar Aug 28 '24 18:08 zanieb

The change in uv 0.4.0 still doesn't fix the scenario when running uv run that will install all --dev dependencies that aren't installed usually in a container. You still need to do uv run --no-project everytime. (think ssh into a container to debug)

ddorian avatar Sep 06 '24 11:09 ddorian

There's uv run --no-dev as well. Might make sense to add UV_NO_DEV or UV_NO_PROJECT for containers. What happens if you delete the pyproject.toml and uv.lock after you setup the project? I presume we won't discover the additional dependencies then.

zanieb avatar Sep 06 '24 12:09 zanieb

I presume we won't discover the additional dependencies then.

Yes, it runs normally.

ddorian avatar Sep 06 '24 13:09 ddorian

There is --frozen but is there a uv run --no-sync? Just something to say don't do anything, just use the venv as it is (but it's not no-project, I'm running w.r.t the current project's workspace and virtualenv)

bluss avatar Sep 06 '24 14:09 bluss

uv run --frozen should be effectively the same as uv run --no-sync

edit: The above is wrong, I misunderstood.

zanieb avatar Sep 06 '24 14:09 zanieb

I thought frozen would be it, but it installs packages if they are missing. To clarify I'm looking for a flag that makes uv run never modify the virtualenv.

Here's a minimal reproducer. Yes I am deleting the venv -- making it empty -- in this example.

uv init --package testrun
# Initialized project `testrun` at `/home/user/testrun`
cd testrun/
uv sync
# Using Python 3.12.3
# Creating virtualenv at: .venv
# ...
uv venv
# Using Python 3.12.3
# Creating virtualenv at: .venv
# Activate with: source .venv/bin/activate
uv run --frozen hello
# Installed 1 package in 1ms
# Hello from testrun!

It would install more (dev-deps, other missing deps) if those existed in the example.

bluss avatar Sep 06 '24 15:09 bluss

uv run --frozen just ensures that we don't update the lock. It still syncs the environment.

charliermarsh avatar Sep 06 '24 15:09 charliermarsh

I want to add a use case. I have a project that's actually not virtual (I do want it to be an installable package). However, PyCharm becomes buggy when I have the package itself (maybe also because it's editable?) as a dependency of an SSH interpreter. So I want to add such a flag (--no-install-*) when I run some uv subcommand from this env.

bryant1410 avatar Nov 18 '24 23:11 bryant1410

poetry allows us to have a custom build.py build script that scripts how simple C-extension Python package is to be built. Even the sckit-build-core build backend allows for CMakeList.txt (i.e. the manifest file for cmake), essentially a build script for CMake. uv's preferred build backend hatch allows for custom build hook with hatch_build.py.

Both poetry's build.py and hatch's hatch_build.py require build dependencies to be installed when the uv build command is run, but I cannot do e.g. uv add --group build hatching without installing the project. But to install the project, I need to first install the build dependencies because of uv insistence on installing the current project when adding dependencies!

Do you see the circle here? 😅

uv needs to allow us to add a dependency without installing the current project so that we can break this cycle (of course I can do pip install hatchling to sidestep this but it completely defeats the purpose of using uv, right?).

There are many use cases for having a custom build script, for one, maybe you want to fetch a C header file as a dependency and generate Cython bindings at build time, using CMake would have been overkill but uv does not currently let us do this with ease either!

jymchng avatar Feb 17 '25 03:02 jymchng

@jymchng

uv add --group build hatchling --no-sync
uv sync --only-group build

zanieb avatar Feb 17 '25 14:02 zanieb

@jymchng

uv add --group build hatchling --no-sync
uv sync --only-group build

Thank you for your reply.

Is there a way to add and install a dependency without building the entire project in the process? Because in development, my build script may be failing because it is incompleted but I need the dependency to be install so I can run and test individual modules.

jymchng avatar Feb 21 '25 03:02 jymchng

I don't follow. I think you may need to open a new issue with a complete, minimal example.

zanieb avatar Mar 01 '25 17:03 zanieb

i'm also interested in this. usecase is:

  1. project has deps and dev deps
  2. project is built and ran in docker
  3. ci builds the docker container and boots it, run pytest against it

for this last part i need to install only pytest in the ci runner but exec of uv run pytest tests/* will download prod deps too

y-nk avatar Apr 04 '25 09:04 y-nk

Another use case: Want to add a dependency to a project that is not installable on the developer platform like macOS, e.g. because missing binaries, built-time requirements etc. and the dependency is only needed in deployment (which is a linux environment and the dependency can be installed). In that case, I want to resolve all dependencies without installing them, in particular, I would like to add sagemaker-training to an optional deployment group.

uv add sagemaker-training --group deployment --no-install-deps-or-similar

and then commit the changes in the uv.lock so I can install that optional group later.

At the moment, I do on my local dev environment:

# 1. manually add dependency to `pyproject.toml`
# 2. sync
uv sync --all-groups --no-group deployment # this updates uv.lock`

# 3. later in the deployment environment
uv install # with some combination of --groups / --no-groups to include deployment group

lorenzwalthert avatar Jun 08 '25 20:06 lorenzwalthert

@lorenzwalthert I think you should use a platform marker instead for that use-case?

zanieb avatar Jun 09 '25 20:06 zanieb

@zanieb , my use case is running a tool (and only that tool with its dependencies) in an isolated venv at a version locked in uv.lock. For example, the tool is bindep, in its own dependency group. I would like to run something like:

uv run --frozen --no-default-groups --group bindep --isolated --no-install-project bindep

I need to use the bindep tool to prepare the build environment, before I am actually able to build/install the actual project.

This seems to be work-around, but not ideal...

uv sync --frozen --no-default-groups --group bindep --no-install-project
uv run --no-sync bindep
rm -r .venv

cavedon avatar Jun 26 '25 19:06 cavedon