uv icon indicating copy to clipboard operation
uv copied to clipboard

Add dependabot support

Open mjclarke94 opened this issue 1 year ago • 11 comments

I'm not quite sure on whether this is a rye or a uv issue, but it would be broadly useful to have dependabot understand rye/uv backed projects as a target for dependabot updates. The current known targets are pip and poetry (which you leave as pip, they just have support for a poetry-style pyproject file).

Whether there is some way of tricking dependabot using the pip compatibility layer, or if it's better to have bespoke handlers I'm not sure!

mjclarke94 avatar Mar 18 '24 15:03 mjclarke94

We don't have a lockfile standard here at uv so I think that the pip dependabot backend should just work? I'm not sure there's more for us to do at this time. This seems more relevant for rye.

zanieb avatar Mar 18 '24 16:03 zanieb

Fair enough! As a secondary point, it might be useful to work with the dependabot maintainers to allow them to use uv for dependency resolution. We have some repositories which dependabot times out trying to solve, and uv solves pretty much instantaneously.

Given they explicitly support poetry (including generating a lock file), there's definitely precedent for it, and the goal of pip compatibility means that this could just be a straight swap in some repositories.

mjclarke94 avatar Mar 20 '24 20:03 mjclarke94

We have some repositories which dependabot times out trying to solve, and uv solves pretty much instantaneously.

Do you have a concrete example?

Not uv related, but I have a significant improvement to Pip's resolution for complex backtracking cases (https://github.com/notatallshaw/pip/tree/prefer-conflicting-causes), and it would be good to know as many real world cases it solves or not.

notatallshaw avatar Mar 21 '24 14:03 notatallshaw

I tried spending a little time on this today and got some feedback from dependabot's CI here https://github.com/dependabot/dependabot-core/actions/runs/9581241755/job/26417616046

One error appears to be easy to fix, there is a check to control that there is an error trying to install a dependency that should fail.

The other I have to look into but here is the message we get from uv when dependabot tries to use it as a direct replacement for pip-compile

"error: invalid value 'pyyaml==6.0.1' for '--upgrade-package <UPGRADE_PACKAGE>': Not a valid package or extra name: \"pyyaml==6.0.1\". Names must start and end with a letter or digit and may only contain -, _, ., and alphanumeric characters.\n\nFor more information, try '--help'."

avilaton avatar Jun 19 '24 11:06 avilaton

Made a bit more progress and found that there is an existing issue here to fix what I encountered above https://github.com/astral-sh/uv/issues/1964, writing it down for future generations.

avilaton avatar Jul 03 '24 19:07 avilaton

The above issue is solved, one step closer! We are now blocked with a tiny header difference issue https://github.com/astral-sh/uv/issues/5031 which is already merged and will be out with next release, thanks @skshetry .

I've mentioned it in another issue but I'm holding from the autogenerated by uv string in the header to swap pip-compile for uv pip compile for the lack of any better idea and since the header still receives updates, I'd like for this to be taken into account.

avilaton avatar Jul 14 '24 14:07 avilaton

Another alternative could to be parse the compile_command. That is what renovatebot does.

skshetry avatar Jul 14 '24 14:07 skshetry

Is there any update on this or an estimate on when uv will support dependabot? I can see the issues you mentioned @avilaton are resolved.

albertferras-vrf avatar Aug 14 '24 13:08 albertferras-vrf

Is there any update on this or an estimate on when uv will support dependabot? I can see the issues you mentioned @avilaton are resolved.

I think the uv side of it is done, but need help adding more test coverage on the dependabot PR I submitted and haven't had time to do it. If you have time please have a look at it, all help is welcome!

avilaton avatar Aug 14 '24 16:08 avilaton

Maybe I'm reading it wrong but this issue seems to have evolved a little after it was originally created. I'm blocked from adopting UV until dependably supports bumping dev dependencies. Is this supported? Is support coming?

danieleades avatar Aug 21 '24 16:08 danieleades

See #6236 — basically it's not supported yet. I think they're working on it still. Renovate supports this.

zanieb avatar Aug 21 '24 18:08 zanieb

👋 from the GitHub Dependabot team. There's a PR in dependabot-core that aims to help support uv. @zanieb in case you didn't notice the tag there, i'm pinging you here to see if you could take a look :)

Thanks!

jonjanego avatar Sep 04 '24 15:09 jonjanego

Noting this just shipped GA: https://github.blog/changelog/2025-03-13-dependabot-version-updates-now-support-uv-in-general-availability/

sstrullmyer avatar Mar 13 '25 19:03 sstrullmyer

Hello from the GitHub Dependabot team 👋

I have made some updates to how we handle uv.lock files and merged today.

We have a generated bump PR for the httpx dependency on our testing repo.

I generated the original uv.lock file on MacOS. The subsequent run of uv lock --upgrade-package httpx in the Dependabot action also removed some wheels for the unrelated dependency charset-normalizer.

Is this expected behaviour? Any feedback on this would be gratefully received. Thank you!

markhallen avatar Mar 19 '25 14:03 markhallen

Thank you for reaching out!

I took a look at that pull request (in Dependabot) and it looks better.

What uv version was the lockfile generated with originally? We may have added additional filtering of wheels that cannot be selected, e.g., as in #12299

zanieb avatar Mar 19 '25 14:03 zanieb

On a separate note, I see you're testing against a dependency that is also pinned in the pyproject.toml

--- a/pyproject.toml
+++ b/pyproject.toml
@@ -5,6 +5,6 @@ description = "Add your description here"
 readme = "README.md"
 requires-python = ">=3.9"
 dependencies = [
-    "httpx==0.20",
+    "httpx==0.28.1",
     "requests==2.31.0",
 ]

I don't think this is a regular pattern, since that's the point of the uv.lock file. It'd be good to test with a constrained version too, e.g., httpx>=0.20.

zanieb avatar Mar 19 '25 15:03 zanieb

It looks like somehow the requires-python range is being changed from >=3.9 to >=3.13 and we're dropping the old wheels. Are you changing the Python version range? If so, why?

zanieb avatar Mar 19 '25 15:03 zanieb

It looks like somehow the requires-python range is being changed from >=3.9 to >=3.13 and we're dropping the old wheels. Are you changing the Python version range? If so, why?

We are not intentionally changing it and actually have code to maintain it in case it is inadvertently updated.

I'm going to look into what is happening here. Thanks

markhallen avatar Mar 19 '25 15:03 markhallen

It seems like this occurs at https://github.com/dependabot/dependabot-core/blob/59943c098d9a28736a2f3ac2945faf3048e662ed/uv/lib/dependabot/uv/file_updater/lock_file_updater.rb#L166-L170

You don't need to do that. The project metadata needs to be preserved for a correct lock.

zanieb avatar Mar 19 '25 15:03 zanieb

For general semantics, uv lock will look at the pyproject.toml and at uv.lock and only make those change required to align the lockfile with pyproject.toml. This means if a ==-pinned dependency is updated in pyproject.toml, uv lock will make the relevant minimal change to uv.lock. uv lock does no change if it doesn't have to, so e.g. a plain httpx requirement won't be updated. uv lock --upgrade-package httpx otoh does update the lockfile entry for a plain httpx requirement to the latest version compatible with the rest of the pyproject.toml.

For the Python version, we use project.requires-python to determine which versions are allowed (their Requires-Python must start at least as low as the main project's one). Separately, we need a Python interpreter during locking for the case that we need to build a package. The default logic for discovering the Python interpreter is multi-layered (different sources for interpreters across platforms), but you can pass a specific one --python.

konstin avatar Mar 19 '25 15:03 konstin

It seems like this occurs at https://github.com/dependabot/dependabot-core/blob/59943c098d9a28736a2f3ac2945faf3048e662ed/uv/lib/dependabot/uv/file_updater/lock_file_updater.rb#L166-L170

You don't need to do that. The project metadata needs to be preserved for a correct lock.

I've raised a PR to remove this.

markhallen avatar Mar 19 '25 15:03 markhallen

Thanks for the comment @konstin

Separately, we need a Python interpreter during locking for the case that we need to build a package. The default logic for discovering the Python interpreter is multi-layered (different sources for interpreters across platforms), but you can pass a specific one --python.

Can we do something similar to uv lock --upgrade-package ... that does not try to build or download the packages and only update the lock file or even just returning the updated lock file contents? This is an approach used for other package managers in Dependabot.

markhallen avatar Mar 20 '25 08:03 markhallen

Technically, you can use --no-build to not build any packages. But this means that you won't be able to resolve any resolutions with packages that only have source distribution (with Metadata Version <2.2, which is still common), so we highly recommend running the regular uv lock --upgrade-package. uv tries hard to just update the lockfile (it's what makes uv fast), but the historic lack of reliable metadata forces us to sometimes build just to extract dependency information.

Usually, we also don't download the entire wheels, given PEP 658 or Range Request support for metadata, but for the (rare) indexes that don't support either, we have no option but to download an entire wheel and read the METADATA from it.

Both of those caveats should be generally the same for pip/pip-tools and poetry (pip for example has --only-binary :all:). We expose those options more prominently, we're the only tool not written in Python, so for us, running Python code is an explicit extra step and not just a subprocess.check_call([sys.executable, ...]).

konstin avatar Mar 20 '25 09:03 konstin

The docs at https://docs.astral.sh/uv/guides/integration/dependency-bots/#dependabot still say this:

Support for uv is not yet available. Progress can be tracked at dependabot/dependabot-core#10478.

I think that's not accurate any more, Dependabot support is available (complete or partially)?

Could you please update the docs page?

cdeil avatar May 22 '25 06:05 cdeil

I would suggest to describe it as partial support. There are still some features missing (e.g. https://github.com/dependabot/dependabot-core/issues/11913) and quite a few open issues (https://github.com/dependabot/dependabot-core/issues?q=is%3Aissue%20state%3Aopen%20uv). But basic support is available and this should be expressed in the docs, because it is likely that some folks won't adopt uv without it (imho)

christianplatta1012 avatar May 22 '25 06:05 christianplatta1012

I recently added it to my project (https://github.com/epicserve/django-base-site/blob/main/.github/dependabot.yml#L16) and haven't encountered any issues yet. My project is a starter Django template, so it's not something that runs in production, and is small compared to a mature project.

epicserve avatar May 22 '25 12:05 epicserve

We updated the section in the documentation about dependabot and added an example how to use it. We still see a number of issue blocking adoption in https://github.com/dependabot/dependabot-core/issues?q=is%3Aissue%20state%3Aopen%20uv, so we'll keep this open a bit longer.

konstin avatar May 28 '25 12:05 konstin

I just tried uv dependabot and it doesn't seem to generate a PR. Am I missing something? https://github.com/kaihendry/pycallfoo/blob/main/.github/dependabot.yml

pycallfoo depends on pyunderstand 0.1.2, but pyunderstand has been updated to v0.1.3

kaihendry avatar Jun 23 '25 09:06 kaihendry

Is this the place to ask questions about uv x dependabot?

From the docs:

Dependabot supports updating uv.lock files.

Why doesn't dependabot also update the pyproject.toml? :) Is that planned?

phaabe avatar Jul 10 '25 12:07 phaabe

That's a question for GitHub and the Dependabot team.

zanieb avatar Jul 10 '25 13:07 zanieb