uv icon indicating copy to clipboard operation
uv copied to clipboard

Support URL requirements in transitive dependencies

Open acostapazo opened this issue 1 year ago • 7 comments

First of all, thank you for this awesome tool, its potential is enormous. 😄

Platform: Ubuntu + MacOs uv version: 0.1.6

I guess this issue is related with https://github.com/astral-sh/uv/issues/1603, but wasn't resolved in v0.1.16. The error occurs when trying to install a package that in turn has a dependency that is installed via URL

For example, If we have a requirement.txt like the following to install a requirement with a URL.

requirement.txt:

packageA @ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageA

And the packageA also have a requirement to install another package (packageB) from URL.

pyproject.toml (from package A)

..
dependencies = [
    "packageB @ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageB"
]
...
flowchart LR

    requirements(requirements.txt)

    subgraph packageA[packageA]
        pyprojectA(pyproject.toml)
    end

    subgraph packageB[packageB]
        pyprojectB(pyproject.toml)
    end

    requirements -.URL requirement.-> packageA
    pyprojectA -.URL requirement.-> packageB
    pyprojectB --> pydantic

When I try to install it with my requirements.txt with uv (v0.1.16) i'm obtaining the following error:

 > uv pip install -r requirements.txt
  Updated https://github.com/alice-biometrics/uv-playground (961d0e8)                                                                                                            
  error: Package `packageb` attempted to resolve via URL: git+https://github.com/alice-biometrics/uv- 
  playground@main#egg=src&subdirectory=issues/url_deps/packageB. URL dependencies must be expressed as direct 
  requirements or constraints. Consider adding `packageb @ git+https://github.com/alice-biometrics/uv- 
  playground@main#egg=src&subdirectory=issues/url_deps/packageB` to your dependencies or constraints file.

Currently, it appears that uv is encountering difficulty resolving inherited dependencies (such as packageB) from URLs specified in the dependencies section of pyproject.toml for packageA. I've provided a reproducible example in the following repository: uv-playground.

When packageB is included directly in the main requirement.txt, uv successfully resolves the dependency. However, I believe uv should automatically resolve inherited dependencies from dependent packages, irrespective of whether they are specified by URL or not, similar to the behavior of pip.

pip example:

 pip install -r requirements.txt   
Collecting packageA@ git+https://****@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageA (from -r requirements.txt (line 1))
  Cloning https://****@github.com/alice-biometrics/uv-playground (to revision main) to /private/var/folders/q6/x7kmvkj1007_pzt6qlcp0yg80000gn/T/pip-install-884jgrjp/packagea_8bc58740c10d4bc8aeaac2793fed3350
  Running command git clone --filter=blob:none --quiet 'https://****@github.com/alice-biometrics/uv-playground' /private/var/folders/q6/x7kmvkj1007_pzt6qlcp0yg80000gn/T/pip-install-884jgrjp/packagea_8bc58740c10d4bc8aeaac2793fed3350
  Resolved https://****@github.com/alice-biometrics/uv-playground to commit 961d0e851f95831440e39af1fbfb65232fc8bad2
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting packageB@ git+https://****@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageB (from packageA@ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageA->-r requirements.txt (line 1))
  Cloning https://****@github.com/alice-biometrics/uv-playground (to revision main) to /private/var/folders/q6/x7kmvkj1007_pzt6qlcp0yg80000gn/T/pip-install-884jgrjp/packageb_1edf5d1949014c61b524fb3b3adab146
  Running command git clone --filter=blob:none --quiet 'https://****@github.com/alice-biometrics/uv-playground' /private/var/folders/q6/x7kmvkj1007_pzt6qlcp0yg80000gn/T/pip-install-884jgrjp/packageb_1edf5d1949014c61b524fb3b3adab146
  Resolved https://****@github.com/alice-biometrics/uv-playground to commit 961d0e851f95831440e39af1fbfb65232fc8bad2
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Requirement already satisfied: pydantic in /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages (from packageB@ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageB->packageA@ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageA->-r requirements.txt (line 1)) (2.6.1)
Requirement already satisfied: annotated-types>=0.4.0 in /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages (from pydantic->packageB@ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageB->packageA@ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageA->-r requirements.txt (line 1)) (0.6.0)
Requirement already satisfied: pydantic-core==2.16.2 in /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages (from pydantic->packageB@ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageB->packageA@ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageA->-r requirements.txt (line 1)) (2.16.2)
Requirement already satisfied: typing-extensions>=4.6.1 in /Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages (from pydantic->packageB@ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageB->packageA@ git+https://@github.com/alice-biometrics/uv-playground@main#egg=src&subdirectory=issues/url_deps/packageA->-r requirements.txt (line 1)) (4.9.0)
Building wheels for collected packages: packageA, packageB
  Building wheel for packageA (pyproject.toml) ... done
  Created wheel for packageA: filename=packageA-0.0.1-py3-none-any.whl size=1493 sha256=f0ac5e361fb53cbfb15361ef254b464aaecb077344a05426f31421afca0bab5e
  Stored in directory: /private/var/folders/q6/x7kmvkj1007_pzt6qlcp0yg80000gn/T/pip-ephem-wheel-cache-q3cf5kxx/wheels/08/bc/d8/e4f479525c376157200e945e40044746e83c025ad0f2d4c467
  Building wheel for packageB (pyproject.toml) ... done
  Created wheel for packageB: filename=packageB-0.0.1-py3-none-any.whl size=1432 sha256=8e04951b1a21171b7fa94b02093832015549d140e9459ca63178f6f919a9d0ae
  Stored in directory: /private/var/folders/q6/x7kmvkj1007_pzt6qlcp0yg80000gn/T/pip-ephem-wheel-cache-q3cf5kxx/wheels/7c/68/71/46e6369994133bcdbc3af801707a51981374abc5de0d7dc830
Successfully built packageA packageB
Installing collected packages: packageB, packageA
Successfully installed packageA-0.0.1 packageB-0.0.1

This issue impacts the ease of managing dependencies and could streamline the workflow significantly if resolved. Thank you for your attention to this matter.

acostapazo avatar Feb 21 '24 13:02 acostapazo

Same error on v0.1.7

acostapazo avatar Feb 22 '24 14:02 acostapazo

I'm not sure, but this behaivor seems to be something by design as you explain at https://github.com/astral-sh/uv/issues/1853.

As @SnoopJ commented, this could be a stopper to our immediate adoption. A flag option would be very nice!

acostapazo avatar Feb 22 '24 14:02 acostapazo

Note that this problem also affects uv pip compile.

One of our transitive dependencies is defined as git+https, which is not ideal but that's life, we don't really want to add it to our requirements because it's an unnecessary burden when updating the root package. This makes uv impossible to use in our project.

Please, consider maintaining this behavior opt-in but having a flag to opt-out of it, otherwise this will be a complete stopper for some use cases.

dcaro21 avatar Feb 23 '24 11:02 dcaro21

This is also going to be a blocker for a lot of people/teams. If there was a way to disable this check, and perhaps others that pip just deals with, that would be a huge plus for bringing this into more teams.

JacobCallahan avatar Mar 01 '24 14:03 JacobCallahan

I'm seeing this with transitive editable dependencies

error: Package `file-utils` attempted to resolve via URL: file:///some/path/file_utils. URL dependencies must be expressed as direct requirements or constraints.

timothyjlaurent avatar Mar 14 '24 17:03 timothyjlaurent

Are transitive editable dependencies possible? How are your dependencies defined?

charliermarsh avatar Mar 14 '24 18:03 charliermarsh

Hi @charliermarsh thanks for the response.

We're currently using poetry with editable dependencies for our monorepo

eg

[tool.poetry.dependencies]
lib1 = {path = "../../lib/lib1", develop = true }

Since any package can have dependencies on others we can have lib1 in this example having dependencies on lib2.

I understand that uv doesn't support poetry dependencies (yet), but we're able to pip install /path/to/a/package and it will install properly (though not respect the pinned deps in the lockfile).

What I'm noticing is that if I uv pip install . it complains not about the first line of deps, but about the second, eg lib2 in this toy example.

timothyjlaurent avatar Mar 15 '24 19:03 timothyjlaurent

The next version of uv will support transitive URL dependencies.

charliermarsh avatar Apr 01 '24 21:04 charliermarsh