Version conflicts installing lock file when multiple git dependencies rely on another git dependency

[conorvtx]

When running

uv pip install -r requirements/lock.txt

I get something like

error: Requirements contain conflicting URLs for package `repo1`:
- git+ssh://[email protected]/org/repo1.git
- git+ssh://[email protected]/org/repo1.git@f6f1bd2f14f887fe63b15632d3a18109ab7a8105

The lock file is generated with

uv pip compile -q --generate-hashes pyproject.toml -o requirements/lock.txt

and has

repo2 @ git+ssh://[email protected]/org/repo2.git@f6f1bd2f14f887fe63b15632d3a18109ab7a8106
    # via current_project (pyproject.toml)

repo3 @ git+ssh://[email protected]/org/repo3.git@f6f1bd2f14f887fe63b15632d3a18109ab7a8104
    # via repo2

repo1 @ git+ssh://[email protected]/org/repo1.git@f6f1bd2f14f887fe63b15632d3a18109ab7a8105
    # via
    #   current_project (pyproject.toml)
    #   repo2
    #   repo3

The pyproject.toml file for current_project contains dependencies like the following

dependencies = [
    "repo1 @ git+ssh://[email protected]/org/repo1.git",
    "repo2 @ git+ssh://[email protected]/org/repo2.git",
]

repo2 and repo3 use setup.pys that have repo1 as a dependency. They require it as follows

# setup.py

def read_requirements(path):
    return [
        line.strip()
        for line in read(path).split("\n")
        if not line.startswith(('"', "#", "-"))
    ]


setup(
    ...
    install_requires=read_requirements("requirements.txt"),
    ...
)

# requirements.txt

repo1 @ git+ssh://[email protected]/org/repo1.git

---

I am surprised I get the error

error: Requirements contain conflicting URLs for package `repo1`:
- git+ssh://[email protected]/org/repo1.git
- git+ssh://[email protected]/org/repo1.git@f6f1bd2f14f887fe63b15632d3a18109ab7a8105

I would expect git+ssh://[email protected]/org/repo1.git@f6f1bd2f14f887fe63b15632d3a18109ab7a8105 to satisfy the more general constraint of git+ssh://[email protected]/org/repo1.git.

Currently to get around this, I have to update the lock file in current_project every time repo1 has a new commit on the main branch.

[charliermarsh]

I think we'd have to special-case this (the "default branch" behavior) if we want to support it. As-is, we resolve both references to a precise SHA, and allow it if the SHAs match.

[conorvtx]

I am surprised that this fails at the install step rather than at the compile step. I would expect that this problem would be surfaced when making the lock file rather than using the lock file. I thought that the lock file would have all the information needed to rebuild a dependency graph as it has the pinned commits of any package installed through git.

[charliermarsh]

I believe that error is occurring during the resolver step.

[charliermarsh]

Perhaps you intend to be using uv pip sync? uv pip install does a resolution of the input requirements.

[charliermarsh]

Honestly that will probably fail too though.

[conorvtx]

actually uv pip sync works! however, i was originally not using sync because it would remove packages installed by conda in the conda environment.

[charliermarsh]

I think what you're trying to do here should probably work (but agree it doesn't).

[NickCrews]

I just ran into this (or a similar flavor) too. IDK if this is significant, but in my case the version is main for both, so it should be even more obvious that the versions are compatible?

error: Requirements contain conflicting URLs for package `noatak`:
- git+https://github.com/ShipCreekGroup/noatak.git@main
- git+https://github.com/ShipCreekGroup/noatak.git@main

[charliermarsh]

Can you come up with a reproduction that I can use to debug? That’s a confusing failure.

[NickCrews]

Hmm, after trying to make out a repro, I think I was running into an issue because one of the remote deps was stale, when I pushed there the error went away. You should ignore me since I'm no longer blocked, I'm sure you have better things to do :) . If it's useful for anyone else, I made a tiny repo here trying to repro, I think that might be a nice template for someone trying to see how multiple libs and transitive deps might interact

[jasongill]

Here is an example:

[project]
name = "example"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "autogen-core==0.4.0dev0",
    "autogen-agentchat==0.4.0dev0",
    "autogen-ext==0.4.0dev0"
]

[tool.uv.sources]
autogen-core = { git = "https://github.com/microsoft/autogen.git", branch = "staging", subdirectory = "python/packages/autogen-core" }
autogen-agentchat = { git = "https://github.com/microsoft/autogen.git", branch = "staging", subdirectory = "python/packages/autogen-agentchat" }
autogen-ext = { git = "https://github.com/microsoft/autogen.git", branch = "staging", subdirectory = "python/packages/autogen-ext" }

as I had the same issue while trying to load an un-published package from Git:

error: Requirements contain conflicting URLs for package `autogen-core` in split `python_full_version == '3.12.*'`:

(note that you'll need to run GIT_LFS_SKIP_SMUDGE=1 uv sync --verbose to reproduce, as there's a different and unrelated error about LFS when uv tries to use git to check the repo out, but that's an issue for another ticket)

[charliermarsh]

Thank you.

[charliermarsh]

@jasongill -- Are you seeing an error like this?

error: Requirements contain conflicting URLs for package `autogen-core`:
- file:///Users/crmarsh/Library/Caches/uv/git-v0/checkouts/2b1782d8bf4da9cf/333c9515/python/packages/autogen-core
- git+https://github.com/microsoft/autogen.git@staging#subdirectory=python/packages/autogen-core

[charliermarsh]

(That looks like an error, but it also looks different from the other traces in this issue.)

[jasongill]

Correct, sorry for the mix-up - I saw the ticket open for "Requirements contain conflicting URLs for package...." and figured it was the same issue

[charliermarsh]

No problem. I will look at it -- feel free to create a separate issue.

[ghidalgo3]

I ran into this problem too, and found that it reproduces when you:

1. Have a pyproject.toml that references multiple git+ dependencies (for example A and B)
2. There are internal dependencies between them (A declares B as a dependency).

The solution is to only reference the top-level one. So in my example, if package A depends on B, then it's not necessary to list B in the list of dependencies of the pyproject.toml.

For reference, this is the dependencies table I've been working on in an internal project:

dependencies = [
    # "pctasks.core @ git+https://github.com/microsoft/planetary-computer-tasks@main#subdirectory=pctasks/core",
    # "pctasks.task @ git+https://github.com/microsoft/planetary-computer-tasks@main#subdirectory=pctasks/task",
    # "pctasks.cli @ git+https://github.com/microsoft/planetary-computer-tasks@main#subdirectory=pctasks/cli",
    # "pctasks.client @ git+https://github.com/microsoft/planetary-computer-tasks@main#subdirectory=pctasks/client",
    "pctasks.run @ git+https://github.com/microsoft/planetary-computer-tasks@main#subdirectory=pctasks/run",
]

pctasks.run references the rest of the projects of pctasks, but if I add them explicitly as dependencies then I will get an error like:

    Updated https://github.com/microsoft/planetary-computer-tasks (a27f47c9413faea76117b205d6da8a86a0bb8d9b)
error: Requirements contain conflicting URLs for package `pctasks-client`:
- file:///private/var/folders/p0/vbz6hyr901z17__1xr798h2w0000gn/T/.tmpYEPdVI/git-v0/checkouts/773f2287cfaa68da/a27f47c/pctasks/client
- git+https://github.com/microsoft/planetary-computer-tasks@main#subdirectory=pctasks/client

I just have to comment on the transitive dependencies to make it work.

[cdb39]

I'm getting a similar issue with uv 0.7.2 and the following setup:

Project B depends on Project A and specifies:

[tool.uv.sources]
A = { git = "ssh://git@github/A.git", rev = "foo" }

Then Project C which has a direct dependency on B and A specifies

[tool.uv.sources]
A = { git = "ssh://git@github/A.git", branch = "foo" }
B = { git = "ssh://git@github/B.git" }

"branch" and "rev" are different but I believe these would actually end up resolving to the same thing.

Personally it feels counter-intuitive that by specifying git as a source, uv is now seemingly transitively using uv sources from that dependency. If I was installing via any other artifact (such as a published sdist), then Project C's resolution would completely ignore Project A's uv.sources, right?

I appreciate that others might actually prefer that uv uses the sources of the dependencies as those dependencies declare, but perhaps the behaviour could be at least configurable? Apologies if it already is, I hunted around but couldn't find anything.

[ships]

I have a similar situation to @cdb39 , it particularly inconvenient when we want to test a feature branch on A, we have to create out a feature branch on B that refers to that branch with no other changes, and change both references upstream. You can imagine this situation cascading though luckily for us it is only 1 deep.

It would be nice to have a configuration for uv in pyproject.toml or even like --no-sources at CLI, which is like --no-transitive-sources or similar.

Thanks for your attention! Lovin this tool.

[ryanc-bs]

Would it be possible to just ignore the [tool.uv.sources] section entirely in dependencies, or else allow the top level pyproject.toml to set a different package source without giving a "Requirements contain conflicting URLs" error?

This is super annoying when installing through git where you have multiple inter-dependent packages if you use git tags and/or branches to specify the version.

For example let's say package A depends on B and C, and B also depends on C. Let's say that in project B the source for C is "https://github.com/foo/[email protected]" and in project A I set the source to ""https://github.com/foo/[email protected]" instead. Currently this results in an error, so I cannot bump the version of C in A without bumping it in B first and then bumping B and C together in A. What I want instead is to just ignore the source defined in B and use A as the source of truth.

[ryanc-bs]

Would it be possible to just ignore the [tool.uv.sources] section entirely in dependencies, or else allow the top level pyproject.toml to set a different package source without giving a "Requirements contain conflicting URLs" error?

This is super annoying when installing through git where you have multiple inter-dependent packages if you use git tags and/or branches to specify the version.

For example let's say package A depends on B and C, and B also depends on C. Let's say that in project B the source for C is "https://github.com/foo/[email protected]" and in project A I set the source to ""https://github.com/foo/[email protected]" instead. Currently this results in an error, so I cannot bump the version of C in A without bumping it in B first and then bumping B and C together in A. What I want instead is to just ignore the source defined in B and use A as the source of truth.

In summary - I'm not familiar with the uv internals but I think the solution here should be quite straightforward from a high level:

With regards to [tool.uv.sources] entries: in case of a conflict, should always use the source which is higher in the tree of dependencies. Only raise an error if two entries at the same level in the hierarchy disagree on the source.

[dnadlinger]

@ryanc-bs Agreed on the usefulness of such an override behaviour: Let's say library A depends on B which is not on PyPI and will require a Git dependency. In the public release of A, I want to depend on an upsteam version of B. For internal development, I want to have a meta-package C that pulls in both A as well as an internal fork of B. The end goal is to have C, with a committed lock file, serve as an internal base development environment, while also keeping A available as a public library.

For this use case, however, it seems that specifying [tool.uv] overide-dependencies = ["B"] in C/pyproject.toml is good enough (?).

[ships]

In our case, i think override-dependencies is sufficient to avoid multiple repo branches! Thanks for that pointer 🎉

It would be ergonomic still, to have something like authoritative = true or override_transitive_sources = true in the tool.uv.sources block, or allow something similar project-wide, since otherwise we would have to replicate the source info re-encoded in the python dependency string syntax, in the override-dependencies.

If i understood, this essentially as @ryanc-bs suggests but with a config flag rather than resolving that way by default.