poetry Example in the Documentation does not work for Torch and TorchVision

Description

My pyproject.toml file (based on documentation )

[project]
authors = [{ name = "rohanmohapatra", email = "censored" }]
description = ""
name = "censored"
readme = "README.md"
requires-python = ">=3.10,<3.13"
version = "0.0.1"

[tool.poetry]
package-mode = false

[project.optional-dependencies]
cpu = ["torch (==2.4.1+cpu)", "torchvision (==0.19.1+cpu)"]
cuda = ["torch (==2.4.1+cu124)", "torchvision (==0.19.1+cu124)"]

[[tool.poetry.source]]
name = "pytorch-cpu"
priority = "explicit"
url = "https://download.pytorch.org/whl/cpu"

[[tool.poetry.source]]
name = "pytorch-cu124"
priority = "explicit"
url = "https://download.pytorch.org/whl/cu124"


[tool.poetry.dependencies]
torch = [
    { markers = "extra == 'cpu' and extra != 'cuda'", version = "2.4.1+cpu", source = "pytorch-cpu" },
    { markers = "extra == 'cuda' and extra != 'cpu'", version = "2.4.1+cu124", source = "pytorch-cu124" },
]
torchvision = [
    { markers = "extra == 'cpu' and extra != 'cuda'", version = "0.19.1+cpu", source = "pytorch-cpu" },
    { markers = "extra == 'cuda' and extra != 'cpu'", version = "0.19.1+cu124", source = "pytorch-cu124" },
]

When I run poetry install --extras "cuda", It shows 2 updates which is exactly what I want

Package operations: 0 installs, 2 updates, 0 removals

  - Updating torch (2.4.1+cpu -> 2.4.1+cu124)
  - Updating torchvision (0.19.1+cpu -> 0.19.1+cu124)

But when I run poetry install --extras "cpu", it shows only 1 update

Package operations: 0 installs, 1 update, 0 removals

  - Downgrading torch (2.4.1+cu124 -> 2.4.1+cpu)

This does not makes sense? Am I doing something wrong?

Workarounds

No workaround yet.

Poetry Installation Method

pip

Operating System

Ubuntu 22.04

Poetry Version

2.1.2

Poetry Configuration

cache-dir = "~/.cache/pypoetry"
data-dir = "~/.local/share/pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
python.installation-dir = "{data-dir}/python" 
repositories.pytorch-cpu.url = "https://download.pytorch.org/whl/cpu"
repositories.pytorch-cu124.url = "https://download.pytorch.org/whl/cu124"
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false

Python Sysconfig

sysconfig.log

Paste the output of 'python -m sysconfig', over this line.

Example pyproject.toml

[project]
authors = [{ name = "rohanmohapatra", email = "censored" }]
description = ""
name = "censored"
readme = "README.md"
requires-python = ">=3.10,<3.13"
version = "0.0.1"

[tool.poetry]
package-mode = false

[project.optional-dependencies]
cpu = ["torch (==2.4.1+cpu)", "torchvision (==0.19.1+cpu)"]
cuda = ["torch (==2.4.1+cu124)", "torchvision (==0.19.1+cu124)"]

[[tool.poetry.source]]
name = "pytorch-cpu"
priority = "explicit"
url = "https://download.pytorch.org/whl/cpu"

[[tool.poetry.source]]
name = "pytorch-cu124"
priority = "explicit"
url = "https://download.pytorch.org/whl/cu124"


[tool.poetry.dependencies]
torch = [
    { markers = "extra == 'cpu' and extra != 'cuda'", version = "2.4.1+cpu", source = "pytorch-cpu" },
    { markers = "extra == 'cuda' and extra != 'cpu'", version = "2.4.1+cu124", source = "pytorch-cu124" },
]
torchvision = [
    { markers = "extra == 'cpu' and extra != 'cuda'", version = "0.19.1+cpu", source = "pytorch-cpu" },
    { markers = "extra == 'cuda' and extra != 'cpu'", version = "0.19.1+cu124", source = "pytorch-cu124" },
]

Poetry Runtime Logs

poetry-runtime.log

Checking keyring availability: Checking if keyring is available
[keyring:keyring.backend] Loading KWallet
[keyring:keyring.backend] Loading SecretService
[keyring:keyring.backend] Loading Windows
[keyring:keyring.backend] Loading chainer
[keyring:keyring.backend] Loading libsecret
[keyring:keyring.backend] Loading macOS
Backend 'fail Keyring' is not suitable
No valid keyring backend was found
Unavailable
Installing dependencies from lock file

Finding the necessary packages for the current system

Package operations: 0 installs, 1 update, 0 removals, 11 skipped

- Downgrading torch (2.4.1+cu124 -> 2.4.1+cpu): Pending...
Multiple source configurations found for download.pytorch.org - pytorch-cpu, pytorch-cu124
Creating new session for download.pytorch.org
Multiple source configurations found for download.pytorch.org - pytorch-cpu, pytorch-cu124
[urllib3:urllib3.connectionpool] Starting new HTTPS connection (1): download.pytorch.org:443
[urllib3:urllib3.connectionpool] https://download.pytorch.org:443 "GET /whl/cpu/torch/ HTTP/1.1" 200 162168
Skipping wheel torch-2.4.1+cpu-cp310-cp310-win_amd64.whl as this is not supported by the current environment
Skipping wheel torch-2.4.1+cpu-cp311-cp311-linux_x86_64.whl as this is not supported by the current environment
Skipping wheel torch-2.4.1+cpu-cp311-cp311-win_amd64.whl as this is not supported by the current environment
Skipping wheel torch-2.4.1+cpu-cp312-cp312-linux_x86_64.whl as this is not supported by the current environment
Skipping wheel torch-2.4.1+cpu-cp312-cp312-win_amd64.whl as this is not supported by the current environment
Skipping wheel torch-2.4.1+cpu-cp38-cp38-linux_x86_64.whl as this is not supported by the current environment
Skipping wheel torch-2.4.1+cpu-cp38-cp38-win_amd64.whl as this is not supported by the current environment
Skipping wheel torch-2.4.1+cpu-cp39-cp39-linux_x86_64.whl as this is not supported by the current environment
Skipping wheel torch-2.4.1+cpu-cp39-cp39-win_amd64.whl as this is not supported by the current environment
- Downgrading torch (2.4.1+cu124 -> 2.4.1+cpu): Installing...
- Downgrading torch (2.4.1+cu124 -> 2.4.1+cpu)
- Installing filelock (3.18.0): Pending...
- Installing filelock (3.18.0): Skipped for the following reason: Already installed
- Installing fsspec (2025.3.2): Pending...
- Installing fsspec (2025.3.2): Skipped for the following reason: Already installed
- Installing jinja2 (3.1.6): Pending...
- Installing jinja2 (3.1.6): Skipped for the following reason: Already installed
- Installing mpmath (1.3.0): Pending...
- Installing mpmath (1.3.0): Skipped for the following reason: Already installed
- Installing networkx (3.4.2): Pending...
- Installing networkx (3.4.2): Skipped for the following reason: Already installed
- Installing numpy (2.2.4): Pending...
- Installing numpy (2.2.4): Skipped for the following reason: Already installed
- Installing sympy (1.13.3): Pending...
- Installing sympy (1.13.3): Skipped for the following reason: Already installed
- Installing markupsafe (3.0.2): Pending...
- Installing markupsafe (3.0.2): Skipped for the following reason: Already installed
- Installing pillow (11.2.1): Pending...
- Installing pillow (11.2.1): Skipped for the following reason: Already installed
- Installing typing-extensions (4.13.2): Pending...
- Installing typing-extensions (4.13.2): Skipped for the following reason: Already installed
- Installing torchvision (0.19.1+cpu): Pending...
- Installing torchvision (0.19.1+cpu): Skipped for the following reason: Already installed

Apr 19 '25 03:04 rohanmohapatra

cf #10120, I don't really believe in negative extras...

Am I doing something wrong?

Yes, you have ignored the big red box right next to the example that you are copying.

Apr 19 '25 08:04 dimbleby

Yes, you have ignored the big red box right next to the example that you are copying.

Isn't that for the first example? This is the second example in that section. Or am I just taking it quite literally? and it applies for both.

Apr 19 '25 20:04 rohanmohapatra

Did you try?

Apr 19 '25 21:04 dimbleby

I did, no luck. And I see another issue, when I add xformers which is compiled for CUDA libs. It starts searching for cu124 packages with/without the re-resolve config. The poetry lock command fails. So, the only feasible workaround I have seen is to

Create two pyproject.toml files.
- One for CPU based systems and the other GPU systems.
- Use diff from Linux to make sure pyproject files are sync.

Apr 19 '25 23:04 rohanmohapatra

works fine either way for me, I assume you are doing something different from what you have told us you are doing.

per your log

- Installing torchvision (0.19.1+cpu): Pending...
- Installing torchvision (0.19.1+cpu): Skipped for the following reason: Already installed

looks like correct behaviour anyway

Apr 19 '25 23:04 dimbleby

works fine either way for me, I assume you are doing something different from what you have told us you are doing.

It isn't installed right? It would be removed if extras=cuda and must be updated when I switch to cpu mode. Something does not feel right there. You can see the Description for it.

Apr 19 '25 23:04 rohanmohapatra

Your own logs show that the expected version of torchvision is installed. That is what "already installed" means.

Apr 19 '25 23:04 dimbleby