pypa
Self-referencing optional dependencies check PyPi version instead of local `pyproject.toml`
Hello,
In my project I'm using optional dependencies for various features. It looks like that:
[project]
name = "libervia-backend"
…
[project.optional-dependencies]
SVG = [
"CairoSVG",
]
GUI = [
"PyQt6"
]
TUI = [
"term-image ~= 0.7.1"
]
remote-control = [
"cbor2"
]
conferences = [
"autobahn ~= 23.6"
]
all = [
"libervia-backend[SVG]",
"libervia-backend[GUI]",
"libervia-backend[TUI]",
"libervia-backend[remote-control]",
"libervia-backend[conferences]"
]
…
[tool.hatch.envs.default]
features = ["all"]
(the whole file is available at https://repos.goffi.org/libervia-backend/file/tip/pyproject.toml#l97)
But when I'm going into dev environment, hatch is looking for those dependencies in the PyPi verion of libervia-backend instead of looking in the definitions above:
…
ERROR: Cannot install cryptography>=41.0.1, libervia-backend[conferences]==0.8.0, oldmemo==1.0.0 and pyopenssl==24.1.0 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested cryptography>=41.0.1
oldmemo 1.0.0 depends on cryptography>=3.3.2
pyopenssl 24.1.0 depends on cryptography<43 and >=41.0.5
libervia-backend[conferences] 0.8.0 depends on cryptography<3.5
(the 0.8.0 version is the one on PyPi and doesn't have such conferences optional dependencies).
Can you please explain how you came to the conclusion that it's looking at PyPI rather than locally? I'm trying to understand.
Hi, thanks for your reply.
It's looking for libervia-backend==0.8 which is the Pypi version, my local version is 0.9.0.dev0, and it says that libervia-backend depends on cryptography<3.5 which was the dependency of version 0.8 (the one on PyPi, cf. https://repos.goffi.org/libervia-backend/file/f97be63871e4/setup.py) while the current pyproject.toml uses cryptography >= 41.0.1). So it really smells like it's checking the PyPi version, but I may have missed something.
@ofek sorry for the late answer, I was busy at a dev sprint last week and I forgot about this issue.
On a machine with Mercurial installed, you can reproduce with this:
% hg clone https://repos.goffi.org/libervia-backend/
% cd libervia-backend
# replace pyproject.toml with a simplified version for testing
% cat """
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "libervia-backend"
description = ""
license = "AGPL-3.0-or-later"
requires-python = ">=3.10"
authors = [
{ name = "Libervia Dev Team", email = "[email protected]" },
]
dependencies = [
"cryptography >= 41.0.1",
]
[project.optional-dependencies]
conferences = [
"autobahn ~= 23.6"
]
all = [
"libervia-backend[conferences]"
]
[tool.hatch.envs.default]
# we need system-package for GPG
system-packages = true
features = ["all"]
[tool.hatch.envs.dev]
dependencies = [
]
[tool.hatch.version]
path = "libervia/backend/__init__.py"
""" > pyproject.toml
% hatch -v -e dev shell
And the result is:
% hatch -v -e dev shell
Finished checking dependencies
Requirement already satisfied: cryptography>=41.0.1 in /usr/lib/python3.12/site-packages (42.0.6)
Collecting libervia-backend[conferences]
Using cached libervia_backend-0.8.0-py3-none-any.whl.metadata (2.1 kB)
Requirement already satisfied: cffi>=1.12 in /usr/lib/python3.12/site-packages (from cryptography>=41.0.1) (1.16.0)
WARNING: libervia-backend 0.8.0 does not provide the extra 'conferences'
Requirement already satisfied: babel<3 in /usr/lib/python3.12/site-packages (from libervia-backend[conferences]) (2.14.0)
Requirement already satisfied: dbus-python<1.3 in /home/goffi/.local/share/hatch/env/virtual/libervia-backend/Yb3zO-f1/dev/lib/python3.12/site-packages (from libervia-backend[conferences]) (1.2.18)
Requirement already satisfied: html2text<2020.2 in /home/goffi/.local/share/hatch/env/virtual/libervia-backend/Yb3zO-f1/dev/lib/python3.12/site-packages (from libervia-backend[conferences]) (2020.1.16)
Requirement already satisfied: jinja2>=2.10.3 in /usr/lib/python3.12/site-packages (from libervia-backend[conferences]) (3.1.4)
Requirement already satisfied: langid<2 in /home/goffi/.local/share/hatch/env/virtual/libervia-backend/Yb3zO-f1/dev/lib/python3.12/site-packages (from libervia-backend[conferences]) (1.1.6)
Requirement already satisfied: lxml>=3.1.0 in /home/goffi/.local/share/hatch/env/virtual/libervia-backend/Yb3zO-f1/dev/lib/python3.12/site-packages (from libervia-backend[conferences]) (5.2.1)
Requirement already satisfied: markdown>=3.0 in /usr/lib/python3.12/site-packages (from libervia-backend[conferences]) (3.6)
Requirement already satisfied: miniupnpc<2.1 in /home/goffi/.local/share/hatch/env/virtual/libervia-backend/Yb3zO-f1/dev/lib/python3.12/site-packages (from libervia-backend[conferences]) (2.0.2)
Requirement already satisfied: mutagen<2 in /usr/lib/python3.12/site-packages (from libervia-backend[conferences]) (1.47.0)
Requirement already satisfied: netifaces<0.12 in /usr/lib/python3.12/site-packages (from libervia-backend[conferences]) (0.11.0)
Requirement already satisfied: pillow>=6.0.0 in /usr/lib/python3.12/site-packages (from libervia-backend[conferences]) (10.4.0)
Requirement already satisfied: progressbar2<3.54 in /home/goffi/.local/share/hatch/env/virtual/libervia-backend/Yb3zO-f1/dev/lib/python3.12/site-packages (from libervia-backend[conferences]) (3.53.3)
INFO: pip is looking at multiple versions of libervia-backend[conferences] to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install cryptography>=41.0.1 and libervia-backend[conferences]==0.8.0 because these package versions have conflicting dependencies.
The conflict is caused by:
The user requested cryptography>=41.0.1
libervia-backend[conferences] 0.8.0 depends on cryptography<3.5
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip to attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts
Finished syncing dependencies
The cryptography<3.5 dependency comes from libervia-backend 0.8 which is the default version available on PyPi (https://pypi.org/project/libervia-backend/).