New python package manager: uv

[rarkins]

New package manager questionnaire

Did you read our documentation on adding a package manager?

• [x] I've read the adding a package manager documentation.

Basics

What's the name of the package manager?

uv

What language(s) does this package manager support?

Python

How popular is this package manager?

This is a new manager, but it feels like it will replace pip or poetry

Does this language have other (competing?) package managers?

poetry, pip

• [x] Yes (give names).
• [ ] No.

What are the big selling points for this package manager?

• performance
• doesn't depend to python
• usage native standard pep-621

Explain how this package manager is different from existing ones

• It was written in a different programming language, which makes it independent of the environment of the Python programming language itself
• It is very performance
• Config file pyproject.toml usage native standard pep-621

Detecting package files

What kind of package files, and names, does this package manager use?

• pyproject.toml - File described dependencies, he usage to generate a lock file. Usage standard pep-621
• requirements.txt - The manager installs the dependencies described in this file (analog lock file). Created after uv pip compile -o requirements.txt

Which fileMatch pattern(s) should Renovate use?

(^|/)pyproject\\.toml$

Do many users need to extend the fileMatch pattern for custom file names?

• [ ] Yes, provide details.
• [x] No.

Is the fileMatch pattern going to get many "false hits" for files that have nothing to do with package management?

No.

Parsing and Extraction

Can package files have "local" links to each other that need to be resolved?

No

Package file parsing method

The package files should be:

• [x] Parsed together (in serial).
• [ ] Parsed independently.

Which format/syntax does the package file use?

• [ ] JSON
• [x] TOML
• [ ] YAML
• [ ] Custom (explain below)

How should we parse the package files?

• [X] Off the shelf parser.
• [ ] Using regex.
• [ ] Custom-parsed line by line.
• [ ] Other.

Does the package file have different "types" of dependencies?

Usage standard pep-621 and cli uf pip compile --extra {type}

• [X] Yes, production and development dependencies.
• [ ] No, all dependencies are treated the same.

List all the sources/syntaxes of dependencies that can be extracted

Don't know

Describe which types of dependencies above are supported and which will be implemented in future

Don't know

Versioning

What versioning scheme does the package file(s) use?

classic semantic version - x.y.z

Does this versioning scheme support range constraints, like ^1.0.0 or 1.x?

• [x] Supports range constraints (for example: ^1.0.0 or 1.x), provide details.
• [ ] No.

Lookup

Is a new datasource required?

• [ ] Yes, provide details.
• [x] No.

Will users want (or need to) set a custom host or custom registry for Renovate's lookup?

• [ ] Yes, provide details.
• [x] No.

Where can Renovate find the custom host/registry?

• [ ] No custom host or registry is needed.
• [ ] In the package file(s), provide details.
• [ ] In some other file inside the repository, provide details.
• [x] User needs to configure Renovate where to find the information, provide details.

Are there any constraints in the package files that Renovate should use in the lookup procedure?

Don't know

• [ ] Yes, there are constraints on the parent language (for example: supports only Python v3.x), provide details.
• [ ] Yes, there are constraints on the parent platform (for example: only supports Linux, Windows, etc.), provide details.
• [ ] Yes, some other kind of constraint, provide details.
• [ ] No constraints.

Will users need the ability to configure language or other constraints using Renovate config?

• [ ] Yes, provide details.
• [x] No.

Artifacts

Does the package manager use a lock file or checksum file?

requirements.txt - lock file

• [x] Yes, uses lock file.
• [ ] Yes, uses checksum file.
• [ ] Yes, uses lock file and checksum file.
• [ ] No lock file or checksum.

Is the locksum or checksum mandatory?

Don't know

• [ ] Yes, locksum is mandatory.
• [ ] Yes, checksum is mandatory.
• [ ] Yes, lock file and checksum are mandatory.
• [ ] No mandatory locksum or checksum.
• [ ] Package manager does not use locksums or checksums.

If lockfiles or checksums are used: what tool and exact commands should Renovate use to update one (or more) package versions in a dependency file?

Comand not found

Package manager cache

Does the package manager use a cache?

https://github.com/astral-sh/uv?tab=readme-ov-file#dependency-caching

• [x] Yes, provide details.
• [ ] No.

If the package manager uses a cache, how can Renovate control the cache?

uv cache clean - delete .cache directory

• [ ] Package manager does not use a cache.
• [x] Controlled via command line interface, provide details.
• [ ] Controlled via environment variables, provide details.

Should Renovate keep a cache?

• [x] Yes, ignore/disable the cache.
• [ ] No.

Generating a lockfile from scratch

Renovate can perform "lock file maintenance" by getting the package manager to generate a lockfile from scratch. Can the package manager generate a lockfile from scratch? uv pip compile -o requirements.txt Must be usage override this command, because compile would like execute with parameters --extra tests --extra linters, this group dependencies (dev/prod).

• [x] Yes, explain which command Renovate should use to generate the lockfile.
• [ ] No, the package manager does not generate a lockfile from scratch.
• [ ] No, the package manager does not use lockfiles.

Other

What else should we know about this package manager?

Originally posted by @staners2 in https://github.com/renovatebot/renovate/discussions/27817#discussioncomment-8731017

[rarkins]

This looks like it could be an alternative "processor" in the pep621 manager: https://github.com/renovatebot/renovate/tree/main/lib/modules/manager/pep621/processors

Does uv identify itself either in the pyproject.toml or requirements.txt?

[github-actions[bot]]

Hi there,

Get your discussion fixed faster by creating a minimal reproduction. This means a repository dedicated to reproducing this issue with the minimal dependencies and config possible.

Before we start working on your issue we need to know exactly what's causing the current behavior. A minimal reproduction helps us with this.

To get started, please read our guide on creating a minimal reproduction.

We may close the discussion if you, or someone else, haven't created a minimal reproduction within two weeks. If you need more time, or are stuck, please ask for help or more time in a comment.

Good luck,

The Renovate team

[staners2]

This looks like it could be an alternative "processor" in the pep621 manager: https://github.com/renovatebot/renovate/tree/main/lib/modules/manager/pep621/processors

Does uv identify itself either in the pyproject.toml or requirements.txt?

Yes. uv usage standard pep621 and identify itself in the pyproject.toml. It creates a lock file (aka requirements.txt, but can be called differently), which I would like to update automatically with changes in pyproject.toml

[staners2]

This project demonstrate expected behaviour: https://github.com/staners2/uv-renovatebot

[rarkins]

Is there always a 1:1 mapping from pyproject.toml -> requirements.txt? Is the requirements.txt in the same directory?

We need to work out how to locate uv projects, either:

• Look for all pyproject.toml files, then check if they have a requirements.txt as lockfile next to it, (standard approach for pep621), or
• Look for any requirements.txt files which have the uv header and then find the source pyproject.toml from there (similar to how our pip-compile works)

[rarkins]

Reproduction forked to https://github.com/renovate-reproductions/27841

[staners2]

We create requirements.txt near pyproject.toml. Although somebody may call it differently and somewhere differently store it. Because file requirements.txt described in the format ${dependency_name}=${version} and for download dependencies executed command pip install -r ${file_name} or uv pip sync -n ${file_name}, he may named differently.

I think named file with all dependencies after uv compile should change in renovate.json

We have one requirements.txt for all environment (prod, stage, test), although best practice says split you dependencies for each environment. But standard doesn't allow storing dependencies for each environments in one file requirements.txt

[rarkins]

It seems like if we want to do this properly then we'd fileMatch for the requirements.txt output files and not the pep621 input files, then trace them back to their source.

We'd also want both the pep621 as well as pip_requirements managers to ignore these respective files if the uv manager was processing them.

[ods]

Using pyproject.toml is just one of uv usage scenarios, similar to pip-tools. Actually pyproject.toml doesn't work well with dependencies split into production/development.

[rarkins]

We should maybe then group it as part of pip-tools if it's a drop-in replacement. Then we need to detect when uv was used and not pip-tools, and run it instead

[sigma67]

It seems the astral.sh folks are focusing on integrating with rye, which now uses uv as its backend for locking and dependency installation

So there's probably significant overlap with #25273 - may be able to avoid two different managers

[karfau]

not sure if this is somehow implicitly covered by the comments above: https://github.com/astral-sh/uv/blob/main/PIP_COMPATIBILITY.md#pip-compile-defaults

By default, uv does not write the compiled requirements to an output file. Instead, uv requires that the user specify an output file explicitly with the -o or --output-file option.

With the options we tend to use for our python projects, renovate ignores files containing -o in the header.

[zanieb]

👋 just to provide some clarity here, we have a uv pip compile command that should be roughly drop-in for pip-tools's requirements.txt. We also provide additional functionality, like a --universal flag for universal locking in that format.

We are also close to stabilizing a uv lock command which creates a uv.lock file from a pyproject.toml.

[mkniewallner]

Although the issue initially discussed the implementation of pip-tools-like locking mechanism, I've started implementing support for the universal lock file in https://github.com/renovatebot/renovate/pull/30756, I hope that's ok. We can also add support for pip-tools style later on, but I believe we'd need to have a way in Renovate to specify that uv should handle requirements.txt instead of pip-tools, as both tools use the same output files.

[not7cd]

@mkniewallner, regarding pip-tools-like locking, it may be worth extending the existing manager. I can explain in depth how it could work, I have left space to do that. But in short, the manager could detect if pip-compile or uv pip-compile was used to generate a given lock.

edit: as stated before by rarkins https://github.com/renovatebot/renovate/issues/27841#issuecomment-2020381196

[mkniewallner]

@mkniewallner, regarding pip-tools-like locking, it may be worth extending the existing manager. I can explain in depth how it could work, I have left space to do that. But in short, the manager could detect if pip-compile or uv pip-compile was used to generate a given lock.

Oh yeah, I guess this would involve parsing the generated lock file to look for a header that hints at the command used?

I did not notice that uv includes the full command that was run to compile dependencies in the header (including uv part as well), I thought it only included the arguments passed to the command.

[renovate-release]

:tada: This issue has been resolved in version 38.41.0 :tada:

The release is available on:

• GitHub release
• npm package (@latest dist-tag)
• 38.41.0

Your semantic-release bot :package::rocket:

[mkniewallner]

For those that originally used this issue to track support for uv pip compile, this is now tracked in a separate issue: https://github.com/renovatebot/renovate/issues/30909.