nox icon indicating copy to clipboard operation
nox copied to clipboard

Support for uv.lock

Open hynek opened this issue 1 year ago • 30 comments

How would this feature be useful?

uv added a cross-platform lock format in 0.3.0. I can see this to become a standard for packages to keep CI stable.

Given that uv sync doesn't have target-installations (yet), it's tricky

Describe the solution you'd like

I think it would be really cool if session.install(".[tests]") automatically discovered a uv.lock file and ran uv sync --extra tests but I could live with session.sync("extras") too.

Either would improve the overall Python DX tremendously.

Describe alternatives you've considered

Hand-craft it, I guess? One has to do manual change directories etc tho.

Anything else?

No response

hynek avatar Aug 25 '24 07:08 hynek

Would https://github.com/astral-sh/uv/pull/6834 support this use case? It adds UV_PROJECT_ENVIRONMENT, I guess, once that lands, assuming nox provides a way to access the virtualenv path, you could implement a session that sets the uv environment to the specified nox location.

One thing to note about this is they expect to add a warning when VIRTUAL_ENV is set in a follow up PR: https://github.com/astral-sh/uv/pull/6864. But even this only warns if UV_PROJECT_ENVRIONMENT and VIRTUAL_ENV don't match. I'll be looking forward to that support happening.

jamesharris-garmin avatar Aug 30 '24 17:08 jamesharris-garmin

JFTR UV_PROJECT_ENVIRONMENT has shipped.

hynek avatar Sep 04 '24 11:09 hynek

seems like we could just make sure to export UV_PROJECT_ENVRIONMENT and set it equal to the virtualenv then you could call uv sync as a run_install call.

jamesharris-garmin avatar Sep 05 '24 04:09 jamesharris-garmin

After experimenting with this it kind of breaks to do this via uv sync and this envrionment variable. uv sync destroys and recreates the environmetn every time if nox modifies it. so we can't just "reuse' the build environment. Maybe a better aproach would be to automatically generate a requirements file from the lock file and re-install it.

jamesharris-garmin avatar Sep 05 '24 18:09 jamesharris-garmin

That seems a bit convoluted, it would make more sense to write up a proper bug report to Astral and ask them – they're usually quite receptive.

What exactly is nox doing there? Could it already be fixed using uv sync --inexact? Also, uv is so fast, venv reuse doesn't matter at all.

hynek avatar Sep 06 '24 05:09 hynek

I haven't had a chance to dig too far into the behavior to see how this is supposed to work. I don't consider it a bug just yet. maybe nox is using a different python version than I expect?

jamesharris-garmin avatar Sep 06 '24 14:09 jamesharris-garmin

nox-poetry does ~~that~~ something similar for poetry.lock, so maybe what we need is an extension like that for uv?

cc @cjolowicz

edgarrmondragon avatar Sep 09 '24 16:09 edgarrmondragon

Tangent: We don't really support plugins here - ideally, you nox would be able to install it's own plugins, rather than assuming people can inject an extra plugin into wherever nox is installed. I'd like to eventually support these better.

henryiii avatar Sep 09 '24 20:09 henryiii

I would like to contribute uv sync. Should I simple create a PR with my ideas, @henryiii?

tonnico avatar Oct 06 '24 19:10 tonnico

Could you describe your ideas first? You can start with a PR if that's easier. I was sort of waiting until the dependency groups and standardize lock file PEPs get resolution.

henryiii avatar Oct 06 '24 19:10 henryiii

Uh, I didn't realize that there is a new PEP[^1] for lock files. I just played around on my local project and this PR is what I've ended with: #857

I'm using the workspace feature from uv. So I need the --package option a lot.

[^1]: PEP 751

tonnico avatar Oct 06 '24 21:10 tonnico

Any news on this?

dariocurr avatar Nov 13 '24 16:11 dariocurr

My current prefered way is to use something like from the cookbook: https://nox.thea.codes/en/stable/cookbook.html#using-a-lockfile

tonnico avatar Nov 19 '24 04:11 tonnico

I think the environment feature will likely be pushed to the next release, but it still is the next major thing I want to work on.

Recipes like the cookbook one should be fine until then.

henryiii avatar Nov 19 '24 05:11 henryiii

Could you describe your ideas first?

As @edgarrmondragon mentioned above, for me the use case I'm looking for is similar to what nox-poetry provides

  • https://github.com/cjolowicz/nox-poetry

Specifically, any time Session.install() is called, the versions to be installed will be constrained to whatever is listed within uv.lock. Often times, we only want/need to install a subset of the dependencies (think, for unit testing we only want to install the main + test dependencies).

This is also where supporting dependency groups would come in

  • #845

So that we could do something like:

def test(session: Session):
    session.install(".", groups=["test"])
    ...

And this would install the main dependencies and the test dependency group that lists pytest, pytest-cov, etc all with locked versions from uv.lock so everything is reproducible and there is no DRY issues (e.g., having to list dependency lists twice between pyproject.toml and noxfile.py).

johnthagen avatar Jan 14 '25 14:01 johnthagen

I mean this is certainly more ergonomic than what I do but I just wrote an installer method that runs uv sync --frozen --inexact with the --extras and --groups flags required to install the combination of dependency groups/extra requirements I need to support the task I need at any given time.

jamesharris-garmin avatar Jan 14 '25 16:01 jamesharris-garmin

As existing work, tox-uv supports this concept for tox

  • https://github.com/tox-dev/tox-uv?tab=readme-ov-file#uvlock-support

johnthagen avatar Jan 28 '25 20:01 johnthagen

Tangent: We don't really support plugins here - ideally, you nox would be able to install it's own plugins, rather than assuming people can inject an extra plugin into wherever nox is installed. I'd like to eventually support these better.

@henryiii is there an issue to track plugin support?

edgarrmondragon avatar Jan 28 '25 21:01 edgarrmondragon

Plugin support is implemented in #881 but it's been stuck waiting for a review since October.

henryiii avatar Jan 28 '25 21:01 henryiii

Plugin support is implemented in https://github.com/wntrblm/nox/pull/881 but it's been stuck waiting for a review since October.

@henryiii Now that the reference MR is merged

  • #881

Do you have recommendations for how can move tighter uv integration for Nox forward? Is a separate nox-uv package similar to nox-poety the preferred way, or, given the growing popularity of uv, would Nox upstream more specific support for uv.lock files?

johnthagen avatar Mar 03 '25 13:03 johnthagen

I think there are two possibilities. One would be to do a plugin, much like nox-poetry. (And that plugin should update examples to show how to use with the current system). That could be done at any time.

The other is work I have planned (but haven't/wont start for a while), which would provide a new "environment" concept, where you could make an environment then define sessions on it. Combining that with lock support makes sense, IMO, and since uv is an integrated backend, it could be natively supported there. What I was thinking was something like this:

@nox.env(path=".venv")
def dev(env: nox.Env) -> None:
    env.install("some", "deps")

@dev.session
def test(session: nox.Session) -> None:
    session.run("pytest")

And native locking support could be something like this:

@nox.env(path=".venv")
def dev(env: nox.Env) -> None:
    env.sync()

...

henryiii avatar Mar 03 '25 17:03 henryiii

Given uv's native support for transient environments is there even any need for sessions at all? I've been able to run Nox with a disabled Python installation using just uv run. You get full venv isolation without any redundant packages. As far as I can tell it all just works out of the box.

d-miketa avatar Mar 06 '25 08:03 d-miketa

@henryiii I've been working on porting my projects from nox-poetry to nox with uv. I wanted to suggest that it could be nice in the Nox docs to have a consolidated "uv Guide". Currently I had to hunt for various bits in formation in a bunch of locations. I'm trying to figure out what the total current "best practices" are for Nox + uv. So far what I've been able to piece together:

  • https://github.com/astral-sh/uv/issues/6579#issuecomment-2465770748
    • Set uv to default venv backend:
nox.options.default_venv_backend = "uv|virtualenv"
  • https://nox.thea.codes/en/stable/cookbook.html#using-a-lockfile

    • Recipe for manually installing via uv's lockfile until a more general solution exists
  • https://github.com/wntrblm/nox/pull/842

    • nox will automatically try to install Python using uv

I'll try to update this post for others as I go in case it's helpful.

johnthagen avatar Mar 21 '25 14:03 johnthagen

The only other one I can think of directly is sometimes you need session.venv_backend == "uv" to check. Most of my noxfiles (see my projects at iscinumpy.dev or on my GitHub profile readme, about 90% use nox) use uv. I mostly avoid lock files, since I mostly work on libraries.

henryiii avatar Mar 21 '25 14:03 henryiii

Another nox uv feature that I discovered:

  • https://github.com/wntrblm/nox/pull/842
    • nox will automatically try to install Python using uv

You cannot yet configure this (e.g., cannot opt out), however, and that is tracked in

  • #944

johnthagen avatar Mar 24 '25 20:03 johnthagen

I have created a small tool that may be useful to others wanting to facilitate using nox with uv. https://pypi.org/project/nox-uv/

dantebben avatar Apr 07 '25 16:04 dantebben

Reminder, you can include the new PEP 723 support in your docs on how to use your plugin. If you have

/// script
dependencies = ["nox>=2025", "nox-uv"]
///

The nox will download your plugin automatically before running!

henryiii avatar Apr 07 '25 16:04 henryiii

I noticed that the new PEP 723 support creates the python environment from whichever python is python in your path. This is a problem for a system where the default python is something like 3.8 which unfortunately is true for my work machine. Do we have an issue to track supporting the requires-python directive in a metadata script block?

jamesharris-garmin avatar Apr 08 '25 21:04 jamesharris-garmin

We need to support the Python range statement first (then it would be easy). https://github.com/wntrblm/nox/issues/814

henryiii avatar Apr 08 '25 21:04 henryiii

alright i'll keep an eye on that ticket then.

jamesharris-garmin avatar Apr 08 '25 22:04 jamesharris-garmin