rye rye sync fails to find nvidia-cublas-cu12 even though uv pip install works

Steps to Reproduce

uv can find this package fine:

~/proj/uv-test ❯ which uv
/home/singhrac/.cargo/bin/uv
~/proj/uv-test ❯ uv venv
Using Python 3.12.1 interpreter at: /home/singhrac/.rye/py/[email protected]/install/bin/python3
Creating virtualenv at: .venv
Activate with: source .venv/bin/activate
~/proj/uv-test ❯ uv pip install nvidia-cublas-cu12==12.4.2.65
Resolved 1 package in 71ms
Downloaded 1 package in 17.49s
Installed 1 package in 10ms
 + nvidia-cublas-cu12==12.4.2.65

Trying the same with rye works (when uv is turned off):

~/proj ❯ rye init rye-test
success: Initialized project in /home/singhrac/proj/rye-test
  Run `rye sync` to get started
[project]
~/proj ❯ cd rye-test
~/proj ❯ vim pyproject.toml
~/proj/rye-test main ?6 ❯ cat pyproject.toml
[project]
name = "rye-test"
version = "0.1.0"
description = "Add your description here"
authors = [
    { name = "Rachit Singh", email = "[email protected]" }
]
dependencies = [
    "jax[cuda12_pip]>=0.4.25",
]
readme = "README.md"
requires-python = ">= 3.8"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.rye]
managed = true
dev-dependencies = []

[tool.hatch.metadata]
allow-direct-references = true

[tool.hatch.build.targets.wheel]
packages = ["src/rye_test"]

[[tool.rye.sources]]
name = "google-jax"
url = "https://storage.googleapis.com/jax-releases/jax_cuda_releases.html"
type = "find-links"
~/proj/rye-test main ?6 ❯ rye config --set-bool behavior.use-uv=false
~/proj/rye-test main ?6 ❯ rye add "jax[cuda12_pip]"
Added jax[cuda12_pip]>=0.4.25 as regular dependency
~/proj/rye-test main ?6 ❯ rye sync
...
Successfully installed jax-0.4.25 jaxlib-0.4.25+cuda12.cudnn89 ml-dtypes-0.3.2 numpy-1.26.4 nvidia-cublas-cu12-12.4.2.65 nvidia-cuda-cupti-cu12-12.4.99 nvidia-cuda-nvcc-cu12-12.4.99 nvidia-cuda-nvrtc-cu12-12.4.99 nvidia-cuda-runtime-cu12-12.4.99 nvidia-cudnn-cu12-8.9.7.29 nvidia-cufft-cu12-11.2.0.44 nvidia-cusolver-cu12-11.6.0.99 nvidia-cusparse-cu12-12.3.0.142 nvidia-nccl-cu12-2.20.5 nvidia-nvjitlink-cu12-12.4.99 opt-einsum-3.3.0 rye-test-0.1.0 scipy-1.12.0
Done!

but when we set rye to use uv, it fails to resolve:

~/proj/rye-test main ?7 ❯ cp pyproject.toml ../test.toml                                                                                                4m 10s
~/proj/rye-test main ?7 ❯ cd ..
~/proj ❯ rm -rf rye-test
~/proj ❯ rye init rye-test
success: Initialized project in /home/singhrac/proj/rye-test
  Run `rye sync` to get started
~/proj ❯ cp test.toml rye-test/pyproject.toml
~/proj ❯ cd rye-test
~/proj/rye-test main ?5 ❯ rye config --set-bool behavior.use-uv=true
~/proj/rye-test main ?5 ❯ rye sync
Initializing new virtualenv in /home/singhrac/proj/rye-test/.venv
Python version: [email protected]
Generating production lockfile: /home/singhrac/proj/rye-test/requirements.lock
  × No solution found when resolving dependencies:
  ╰─▶ Because only nvidia-cublas-cu12<12.3.4.1 is available and jax[cuda12-pip]==0.4.25 depends on nvidia-cublas-cu12>=12.3.4.1, we can conclude that
      jax[cuda12-pip]==0.4.25 cannot be used.
      And because only jax[cuda12-pip]<=0.4.25 is available and you require jax[cuda12-pip]>=0.4.25, we can conclude that the requirements are unsatisfiable.
error: could not write production lockfile for project

Caused by:
    failed to generate lockfile

Expected Result

The middle outcome, where rye sync succeeds.

Actual Result

Last outcome above, i.e. "failed to generate lockfile".

Version Info

~/proj/rye-test main ?5 ❯ rye --version
rye 0.29.0
commit: 0.29.0 (42b179f36 2024-03-11)
platform: linux (x86_64)
self-python: [email protected]
symlink support: true
uv enabled: true

and separately

~/proj/rye-test main ?5 ❯ uv version
uv 0.1.18

Maybe it's because rye's version of uv is a few weeks old? I think uv is now on 0.1.18 and rye uses 0.1.7? Oh, that seems to be a typo in CHANGELOG.md, since previously it was on 0.1.15. I'll submit a PR for that.

Stacktrace

No response

Mar 13 '24 19:03 rachtsingh

By the way, I ran into this when trying to check if https://github.com/astral-sh/rye/issues/745 is fixed - I think this is essentially the remaining issue.

Mar 13 '24 19:03 rachtsingh

Your pyproject has requires-python >= 3.8, could you try to change that and try again?

I see uv failing that way too

echo 'jax[cuda12_pip]>=0.4.25' | uv pip compile -p 3.8  -  -f 'https://storage.googleapis.com/jax-releases/jax_cuda_releases.html'
warning: The requested Python version 3.8 is not available; 3.10.12 will be used to build dependencies instead.
  × No solution found when resolving dependencies

while it succeeds for python 3.9 or later.

With that said, I'm not sure that's what actually happens in rye.. is it using requires python that way? should it?

FWIW, I have the same version of rye and don't have this problem with your pyproject.toml.

Mar 13 '24 22:03 bluss

Same issue:

~/proj/rye-test main ?5 ❯ cat pyproject.toml                                                                                                                                    8s
[project]
name = "rye-test"
version = "0.1.0"
description = "Add your description here"
authors = [
    { name = "Rachit Singh", email = "[email protected]" }
]
dependencies = [
    "jax[cuda12_pip]>=0.4.25",
]
readme = "README.md"
requires-python = ">= 3.12"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.rye]
managed = true
dev-dependencies = []

[tool.hatch.metadata]
allow-direct-references = true

[tool.hatch.build.targets.wheel]
packages = ["src/rye_test"]

[[tool.rye.sources]]
name = "google-jax"
url = "https://storage.googleapis.com/jax-releases/jax_cuda_releases.html"
type = "find-links"
~/proj/rye-test main ?5 ❯ rye sync
Initializing new virtualenv in /home/singhrac/proj/rye-test/.venv
Python version: [email protected]
Generating production lockfile: /home/singhrac/proj/rye-test/requirements.lock
  × No solution found when resolving dependencies:
  ╰─▶ Because only nvidia-cublas-cu12<12.3.4.1 is available and jax[cuda12-pip]==0.4.25 depends on nvidia-cublas-cu12>=12.3.4.1, we can conclude that jax[cuda12-pip]==0.4.25
      cannot be used.
      And because only jax[cuda12-pip]<=0.4.25 is available and you require jax[cuda12-pip]>=0.4.25, we can conclude that the requirements are unsatisfiable.
error: could not write production lockfile for project

Caused by:
    failed to generate lockfile

You're not able to reproduce the issue? Have you checked that rye is configured to use uv (see my note above about non-uv usage succeeding), and that the pip cache is empty?

Mar 14 '24 03:03 rachtsingh

Not sure how to ensure pip cache is empty, but it still succeeds in the resolution like this -- using your pyproject.toml.

XDG_CACHE_HOME=$PWD UV_NO_CACHE=true rye lock
Initializing new virtualenv in /home/user/ryes/rye-test/.venv
Python version: [email protected]
Generating production lockfile: /home/user/ryes/rye-test/requirements.lock
Generating dev lockfile: /home/user/ryes/rye-test/requirements-dev.lock
Done!

rye 0.29.0
commit: 0.29.0 (42b179f36 2024-03-11)
platform: linux (x86_64)
self-python: [email protected]
symlink support: true
uv enabled: true

requirements.lock

# generated by rye
# use `rye lock` or `rye sync` to update this lockfile
#
# last locked with the following flags:
#   pre: false
#   features: []
#   all-features: false
#   with-sources: false

-e file:.
jax==0.4.25
    # via rye-test
jaxlib==0.4.25+cuda12.cudnn89
    # via jax
ml-dtypes==0.3.2
    # via jax
    # via jaxlib
numpy==1.26.4
    # via jax
    # via jaxlib
    # via ml-dtypes
    # via opt-einsum
    # via scipy
nvidia-cublas-cu12==12.4.2.65
    # via jax
    # via nvidia-cudnn-cu12
    # via nvidia-cusolver-cu12
nvidia-cuda-cupti-cu12==12.4.99
    # via jax
nvidia-cuda-nvcc-cu12==12.4.99
    # via jax
nvidia-cuda-nvrtc-cu12==12.4.99
    # via nvidia-cudnn-cu12
nvidia-cuda-runtime-cu12==12.4.99
    # via jax
nvidia-cudnn-cu12==8.9.7.29
    # via jax
nvidia-cufft-cu12==11.2.0.44
    # via jax
nvidia-cusolver-cu12==11.6.0.99
    # via jax
nvidia-cusparse-cu12==12.3.0.142
    # via jax
    # via nvidia-cusolver-cu12
nvidia-nccl-cu12==2.20.5
    # via jax
nvidia-nvjitlink-cu12==12.4.99
    # via jax
    # via nvidia-cusolver-cu12
    # via nvidia-cusparse-cu12
opt-einsum==3.3.0
    # via jax
scipy==1.12.0
    # via jax
    # via jaxlib

Mar 14 '24 18:03 bluss

I'm at a loss, I have no idea why this would succeed for you but not for me. Maybe I need a fresh install of rye? I've been using rye self update.

Mar 15 '24 00:03 rachtsingh

I'm also having this issue when trying to install tensorflow[and-cuda]==2.16.1, I get the error:

error: Failed to run uv compile   × No solution found when resolving dependencies:
  ╰─▶ Because there is no version of nvidia-cublas-cu12==12.3.4.1 and
      you require nvidia-cublas-cu12==12.3.4.1, we can conclude that the
      requirements are unsatisfiable.
. uv exited with status: exit code: 1

Even though it's right here: https://pypi.org/project/nvidia-cublas-cu12/12.3.4.1/

Apr 16 '24 20:04 God-damnit-all

I'm also having this issue when trying to install tensorflow[and-cuda]==2.16.1, I get the error:
error: Failed to run uv compile   × No solution found when resolving dependencies:
  ╰─▶ Because there is no version of nvidia-cublas-cu12==12.3.4.1 and
      you require nvidia-cublas-cu12==12.3.4.1, we can conclude that the
      requirements are unsatisfiable.
. uv exited with status: exit code: 1
Even though it's right here: https://pypi.org/project/nvidia-cublas-cu12/12.3.4.1/

nvidia cublas doesnt support python 3.12, unfortunately. Pip's error messaging is confusing because it raises that error when there's no version available matching your current Python

Jul 09 '24 19:07 ash-sony

rye rye copied to clipboard

rye sync fails to find nvidia-cublas-cu12 even though uv pip install works

Steps to Reproduce

Expected Result

Actual Result

Version Info

Stacktrace

rye
rye copied to clipboard