tox
tox copied to clipboard
tox-dev
Tox fails to install dependency groups with self references
Issue
I have a project that has a "test" dependency group which installs all the extras from the project being tested. However, because tox has chosen to install dependency groups separately, pip cannot find the dependencies because the project itself is not on the PyPI.
The fix is to install all the dependencies and the project itself in the same step.
Environment
Provide at least:
- OS: Fedora Linux 41
Output of pip list of the host Python, where tox is installed
Package Version
--------------------------------------- -----------
adal 1.2.7
antlr4-python3-runtime 4.13.2
applicationinsights 0.11.9
argcomplete 3.6.2
argopt 0.9.1
attrs 25.1.0
awscli 2.25.0
awscrt 0.24.2
azure-appconfiguration 1.1.1
azure-batch 14.2.0
azure-common 1.1.28
azure-core 1.30.2
azure-cosmos 3.2.0
azure-data-tables 12.4.0
azure-datalake-store 0.0.53
azure-graphrbac 0.60.0
azure-keyvault 1.1.0
azure-keyvault-administration 4.4.0b2
azure-keyvault-certificates 4.7.0
azure-keyvault-keys 4.9.0b3
azure-keyvault-secrets 4.7.0
azure-loganalytics 0.1.0
azure-mgmt-advisor 9.0.0
azure-mgmt-apimanagement 4.0.0
azure-mgmt-appconfiguration 3.0.0
azure-mgmt-appcontainers 2.0.0
azure-mgmt-applicationinsights 1.0.0
azure-mgmt-authorization 4.0.0
azure-mgmt-batch 17.3.0
azure-mgmt-batchai 7.0.0b1
azure-mgmt-billing 6.0.0
azure-mgmt-botservice 2.0.0b3
azure-mgmt-cdn 12.0.0
azure-mgmt-cognitiveservices 13.5.0
azure-mgmt-compute 31.0.0
azure-mgmt-consumption 2.0.0
azure-mgmt-containerinstance 10.1.0
azure-mgmt-containerregistry 10.3.0
azure-mgmt-containerservice 31.0.0
azure-mgmt-core 1.3.2
azure-mgmt-cosmosdb 9.5.1
azure-mgmt-databoxedge 1.0.0
azure-mgmt-datalake-analytics 0.2.1
azure-mgmt-datalake-store 0.5.0
azure-mgmt-datamigration 10.0.0
azure-mgmt-devtestlabs 4.0.0
azure-mgmt-dns 8.0.0
azure-mgmt-eventgrid 10.2.0b2
azure-mgmt-eventhub 10.1.0
azure-mgmt-extendedlocation 1.0.0b2
azure-mgmt-hdinsight 9.0.0
azure-mgmt-imagebuilder 1.3.0
azure-mgmt-iotcentral 10.0.0b1
azure-mgmt-iothub 3.0.0
azure-mgmt-iothubprovisioningservices 1.1.0
azure-mgmt-keyvault 10.3.0
azure-mgmt-kusto 0.3.0
azure-mgmt-loganalytics 13.0.0b4
azure-mgmt-managedservices 1.0.0
azure-mgmt-managementgroups 1.0.0
azure-mgmt-maps 2.0.0
azure-mgmt-marketplaceordering 1.1.0
azure-mgmt-media 9.0.0
azure-mgmt-monitor 5.0.0
azure-mgmt-msi 7.0.0
azure-mgmt-netapp 10.1.0
azure-mgmt-policyinsights 1.1.0b4
azure-mgmt-privatedns 1.0.0
azure-mgmt-rdbms 10.2.0b17
azure-mgmt-recoveryservices 3.0.0
azure-mgmt-recoveryservicesbackup 9.1.0
azure-mgmt-redhatopenshift 1.4.0
azure-mgmt-redis 14.3.0
azure-mgmt-relay 0.1.0
azure-mgmt-resource 23.1.1
azure-mgmt-search 9.0.0
azure-mgmt-security 6.0.0
azure-mgmt-servicebus 8.2.0
azure-mgmt-servicefabric 2.1.0
azure-mgmt-servicefabricmanagedclusters 2.0.0b6
azure-mgmt-servicelinker 1.2.0b2
azure-mgmt-signalr 2.0.0b1
azure-mgmt-sql 4.0.0b17
azure-mgmt-sqlvirtualmachine 1.0.0b5
azure-mgmt-storage 21.2.0
azure-mgmt-synapse 2.1.0b5
azure-mgmt-trafficmanager 1.0.0
azure-mgmt-web 7.2.0
azure-multiapi-storage 1.2.0
azure-storage-common 1.4.2
azure-synapse-accesscontrol 0.5.0
azure-synapse-artifacts 0.19.0
azure-synapse-managedprivateendpoints 0.4.0
azure-synapse-spark 0.2.0
bcrypt 4.1.2
Beaker 1.12.1
beautifulsoup4 4.13.4
blivet 3.11.0
blivet-gui 2.6.0
blockbuster 1.5.23
boto3 1.38.36
botocore 1.38.36
Brlapi 0.8.5
build 1.2.2.post1
CacheControl 0.14.0
cachetools 5.5.2
certifi 2024.8.30
cffi 1.17.1
cfgv 3.4.0
chardet 5.2.0
charset-normalizer 3.4.0
cleo 2.1.0
click 8.1.7
colorama 0.4.6
coverage 7.6.10
crashtest 0.4.1
crypt_r 3.13.1
cryptography 43.0.3
cupshelpers 1.0
dasbus 1.7
dbus-python 1.3.2
decorator 5.1.1
Deprecated 1.2.15
distlib 0.3.9
distro 1.9.0
dnf 4.23.0
docutils 0.20.1
dulwich 0.21.7
exceptiongroup 1.2.2
fabric 3.2.2
fastjsonschema 2.20.0
fedora-third-party 0.10
file-magic 0.4.0
filelock 3.16.1
fluidity-sm 0.2.0
forbiddenfruit 0.1.4
fros 1.1
git-fame 2.0.1
gpg 1.23.2
h11 0.16.0
hatch 1.14.1
hatch-containers 0.7.0
hatchling 1.27.0
httpx 0.28.1
humanfriendly 10.0
humanize 4.9.0
hyperlink 21.0.0
hypothesis 6.124.7
identify 2.6.1
idna 3.10
iniconfig 2.0.0
installer 0.7.0
invoke 2.2.0
iso639 0.1.4
isodate 0.6.1
jaraco.classes 3.4.0
javaproperties 0.8.1
jeepney 0.8.0
jmespath 1.0.1
jsondiff 2.2.1
jsonschema 4.19.1
jsonschema-specifications 2023.11.2
keyring 24.3.1
knack 0.12.0
langtable 0.0.68
lexicon 2.0.1
libcomps 0.1.21
libdnf 0.74.0
louis 3.33.0
lxml 5.3.2
Mako 1.2.3
markdown-it-py 3.0.0
MarkupSafe 2.1.5
mdurl 0.1.2
more-itertools 10.5.0
msal 1.32.3
msal-extensions 1.3.1
msgpack 1.1.0
msrest 0.7.1
msrestazure 0.6.4
mypy 1.15.0
mypy-extensions 1.0.0
nftables 0.1
nodeenv 1.9.1
oauthlib 3.2.2
olefile 0.47
outcome 1.3.0.post0
packaging 24.2
paramiko 3.5.1
Paste 3.10.1
pathspec 0.12.1
perf 0.1
pexpect 4.9.0
pid 2.2.3
pillow 11.0.0
pip 25.1.1
pkginfo 1.11.2
platformdirs 4.3.6
pluggy 1.5.0
ply 3.11
poetry 1.8.4
poetry-core 1.9.1
poetry-plugin-export 1.8.0
portalocker 2.10.1
pre_commit 4.2.0
productmd 1.45
prompt_toolkit 3.0.41
psutil 5.9.8
ptyprocess 0.7.0
pwquality 1.4.5
pyasn1 0.6.0
pycairo 1.25.1
pycparser 2.22
pycrypto 2.6.1
pycups 2.0.4
pycurl 7.45.3
pyenchant 3.2.2
PyGithub 2.3.0
Pygments 2.18.0
PyGObject 3.48.2
pyinotify 0.9.6
PyJWT 2.8.0
pykickstart 3.58
PyNaCl 1.5.0
pyOpenSSL 24.2.1
pyparsing 3.1.2
pyparted 3.13.0
pyproject-api 1.8.0
pyproject_hooks 1.2.0
PySocks 1.7.1
pytest 8.3.4
python-augeas 1.1.0
python-dateutil 2.8.2
python-linux-procfs 0.7.3
python-meh 0.52
python-pam 2.0.2
pyudev 0.24.3
pyxdg 0.27
PyYAML 6.0.1
pyynl 0.0.1
RapidFuzz 3.10.0
referencing 0.36.2
regex 2024.11.6
requests 2.32.3
requests-file 2.0.0
requests-ftp 0.3.1
requests-oauthlib 1.3.1
requests-toolbelt 1.0.0
rich 13.9.4
rpds-py 0.19.1
rpm 4.20.1
ruamel.yaml 0.18.6
ruamel.yaml.clib 0.2.12
s3transfer 0.13.0
scp 0.14.5
SecretStorage 3.3.3
selinux 3.7
semver 3.0.2
sepolicy 3.7
setools 4.5.1
setuptools 69.2.0
shellingham 1.5.4
simpleaudio 1.0.4
simpleline 1.9.0
six 1.16.0
sniffio 1.3.1
sortedcontainers 2.4.0
sos 4.9.1
soupsieve 2.7
sshtunnel 0.4.0
systemd-python 235
tabulate 0.9.0
Tempita 0.5.2
tomli_w 1.2.0
tomlkit 0.13.2
tox 4.27.0
tqdm 4.67.1
trio 0.30.0
trove-classifiers 2024.10.16
trustme 1.2.1
truststore 0.10.0
typing_extensions 4.12.2
urllib3 2.2.3
userpath 1.9.2
uv 0.7.8
uvloop 0.21.0
virtualenv 20.31.2
wcwidth 0.2.13
websocket-client 1.7.0
wrapt 1.17.1
xkbregistry 0.3
xmltodict 0.13.0
zipp 3.20.0
zstandard 0.23.0
Output of running tox
Output of tox -rvv
tox -e py310
py310: install_dependency-groups> python -I -m pip install 'anyio-http[brotli,cbor,json,opentelemetry,zstd]' 'anyio[trio]' 'coverage[toml]>=7' 'hypercorn>=0.17.3' 'pytest>=8.0' 'sse-starlette>=2.3.6' 'starlette[full]>=0.47.1' trustme
Collecting coverage>=7 (from coverage[toml]>=7)
Using cached coverage-7.9.2-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (8.9 kB)
Collecting hypercorn>=0.17.3
Using cached hypercorn-0.17.3-py3-none-any.whl.metadata (5.4 kB)
Collecting pytest>=8.0
Using cached pytest-8.4.1-py3-none-any.whl.metadata (7.7 kB)
Collecting sse-starlette>=2.3.6
Using cached sse_starlette-2.4.1-py3-none-any.whl.metadata (10 kB)
Collecting starlette>=0.47.1 (from starlette[full]>=0.47.1)
Using cached starlette-0.47.1-py3-none-any.whl.metadata (6.2 kB)
Collecting trustme
Using cached trustme-1.2.1-py3-none-any.whl.metadata (5.7 kB)
ERROR: Could not find a version that satisfies the requirement anyio-http[brotli,cbor,json,opentelemetry,zstd] (from versions: none)
ERROR: No matching distribution found for anyio-http[brotli,cbor,json,opentelemetry,zstd]
py310: exit 1 (1.48 seconds) /home/alex/workspace/anyio-http> python -I -m pip install 'anyio-http[brotli,cbor,json,opentelemetry,zstd]' 'anyio[trio]' 'coverage[toml]>=7' 'hypercorn>=0.17.3' 'pytest>=8.0' 'sse-starlette>=2.3.6' 'starlette[full]>=0.47.1' trustme pid=588857
py310: FAIL code 1 (1.49 seconds)
evaluation failed :( (1.74 seconds)
Minimal example
[build-system]
requires = ["setuptools >= 77"]
build-backend = "setuptools.build_meta"
[project]
name = "exampleproject"
version = "1.0.0"
[project.optional-dependencies]
json = ["orjson >= 3.10.18"]
[dependency-groups]
test = ["exampleproject[json]"]
[tool.tox]
env_list = ["py313"]
requires = ["tox >= 4.22"]
[tool.tox.env_run_base]
package = "editable"
dependency_groups = ["test"]
Installation works fine with: pip install --group test -e .
I didn't include the output of tox -rvv as it exceeded the allowed body size.
I'm not sure that's even relevant, given the output I did include, plus the minimal repro.
Hey, I also ran into this issue and created a minimal reproducer, here's the link https://github.com/karlicoss/tox_dependencygroup_self_reference
It's often useful to install some optional dependencies while doing testing/linting in the project.
Test dependencies are usually declared as dependency-group, and you can refer to projects' optional dependencies via "projectname[optional-dependency-name]" syntax. E.g. here's example of pdm doing a similar thing.
Current behaviour
However, tox installs dependency groups constituents directly, instead of using --group argument of pip:
$ uv tool run tox -e test --recreate
test: remove tox env folder /code/playground_tox/tox_dependencygroup_self_reference/.tox/test
.pkg: remove tox env folder /code/playground_tox/tox_dependencygroup_self_reference/.tox/.pkg
test: install_dependency-groups> python -I -m pip install pytest 'tox_dependencygroup_self_reference[optional]'
...
ERROR: Could not find a version that satisfies the requirement tox_dependencygroup_self_reference[optional] (from versions: none)
ERROR: No matching distribution found for tox_dependencygroup_self_reference[optional]
As a result, pip doesn't know of tox_dependencygroup_self_reference and fails.
Expected behaviour
The correct approach is probably using something like this instead:
.tox/test/bin/pip install --group testing -e .
After that the transitive optional dependencies are installed as expected:
$ .tox/test/bin/pip list
Package Version Editable project location
---------------------------------- ---------- -------------------------------------------------------
certifi 2025.11.12
charset-normalizer 3.4.4
...
pytest 9.0.2
requests 2.32.5
tox_dependencygroup_self_reference 0.1.1 /code/playground_tox/tox_dependencygroup_self_reference
urllib3 2.6.2
and test passes:
.tox/test/bin/python -m pytest test_main.py
