tox
tox copied to clipboard
Published
20 hours ago •
tox-dev
tox-dev
set_env substiution regression in tox 4.x
Issue
In tox 4.x, with a set_env key that contains a substitution from another section as well as an environment variable substitution, the values that should have been substituted from another section are lost. For more information, see the example below.
Environment
Provide at least:
OS: Linux
`pip list` of the host Python where `tox` is installed:
Package Version
----------------------------- --------------------
aiohttp 3.8.4
aiosignal 1.3.1
alabaster 0.7.13
argon2-cffi 21.3.0
argon2-cffi-bindings 21.2.0
async-timeout 4.0.2
attrs 22.2.0
Babel 2.12.1
beautifulsoup4 4.12.0
build 0.10.0
cachetools 5.3.0
certifi 2022.12.7
cffi 1.15.1
chardet 5.1.0
charset-normalizer 3.1.0
check-manifest 0.49
click 8.1.3
click-default-group 1.2.2
colorama 0.4.6
covdefaults 2.3.0
coverage 7.2.2
defusedxml 0.7.1
devpi-client 6.0.3
devpi-common 3.7.0
devpi-process 0.3.0
devpi-server 6.8.0
diff-cover 7.5.0
distlib 0.3.6
docutils 0.19
editables 0.3
exceptiongroup 1.1.1
execnet 1.9.0
filelock 3.10.7
flaky 3.7.0
frozenlist 1.3.3
furo 2023.3.27
gitdb 4.0.10
GitPython 3.1.31
hatch-vcs 0.3.0
hatchling 1.14.0
hupper 1.12
idna 3.4
imagesize 1.4.1
incremental 22.10.0
iniconfig 2.0.0
itsdangerous 2.1.2
Jinja2 3.1.2
lazy 1.5
MarkupSafe 2.1.2
multidict 6.0.4
packaging 23.0
passlib 1.7.4
PasteDeploy 3.0.1
pathspec 0.11.1
pep517 0.13.0
pip 23.0.1
pkginfo 1.9.6
plaster 1.1.2
plaster-pastedeploy 1.0.1
platformdirs 3.2.0
pluggy 1.0.0
psutil 5.9.4
py 1.11.0
pycparser 2.21
Pygments 2.14.0
pyproject_api 1.5.1
pyproject_hooks 1.0.0
pyramid 2.0.1
pytest 7.2.2
pytest-cov 4.0.0
pytest-mock 3.10.0
pytest-xdist 3.2.1
python-dateutil 2.8.2
re-assert 1.1.0
regex 2023.3.23
repoze.lru 0.7
requests 2.28.2
ruamel.yaml 0.17.21
ruamel.yaml.clib 0.2.7
setuptools 67.5.1
setuptools-scm 7.1.0
six 1.16.0
smmap 5.0.0
snowballstemmer 2.2.0
soupsieve 2.4
Sphinx 6.1.3
sphinx_argparse_cli 1.11.0
sphinx_autodoc_typehints 1.22
sphinx-basic-ng 1.0.0b1
sphinx-copybutton 0.5.1
sphinx_inline_tabs 2022.1.2b11
sphinxcontrib-applehelp 1.0.4
sphinxcontrib-devhelp 1.0.2
sphinxcontrib-htmlhelp 2.0.1
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 1.0.3
sphinxcontrib-serializinghtml 1.1.5
sphinxcontrib-towncrier 0.3.2a0
strictyaml 1.7.3
time-machine 2.9.0
tomli 2.0.1
towncrier 22.12.0
tox 4.4.9.dev3+g8a7327c1
translationstring 1.4
trove-classifiers 2023.3.9
typing_extensions 4.5.0
urllib3 1.26.15
venusian 3.0.0
virtualenv 20.21.0
waitress 2.1.2
WebOb 1.8.7
wheel 0.40.0
yarl 1.8.2
zope.deprecation 5.0
zope.interface 6.0
Output of running tox
Provide the output of `tox -rvv`:
testenv: 269 W remove tox env folder /home/dharding/code/tox_test/.tox/testenv [tox/tox_env/api.py:322]
.pkg: 283 W remove tox env folder /home/dharding/code/tox_test/.tox/.pkg [tox/tox_env/api.py:322]
testenv: 305 I find interpreter for spec PythonSpec(path=/home/dharding/code/opensource/tox/.tox/dev/bin/python) [virtualenv/discovery/builtin.py:56]
testenv: 305 D discover exe for PythonInfo(spec=CPython3.10.10.final.0-64, exe=/home/dharding/code/opensource/tox/.tox/dev/bin/python, platform=linux, version='3.10.10 (main, Mar 25 2023, 18:12:59) [GCC 12.2.1 20230121]', encoding_fs_io=utf-8-utf-8) in /usr [virtualenv/discovery/py_info.py:437]
testenv: 305 D filesystem is case-sensitive [virtualenv/info.py:24]
testenv: 305 D got python info of /usr/bin/python3.10 from /home/dharding/.local/share/virtualenv/py_info/1/8a94588eda9d64d9e9a351ab8144e55b1fabf5113b54e67dd26a8c27df0381b3.json [virtualenv/app_data/via_disk_folder.py:129]
testenv: 333 I proposed PythonInfo(spec=CPython3.10.10.final.0-64, system=/usr/bin/python3.10, exe=/home/dharding/code/opensource/tox/.tox/dev/bin/python, platform=linux, version='3.10.10 (main, Mar 25 2023, 18:12:59) [GCC 12.2.1 20230121]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:63]
testenv: 334 D accepted PythonInfo(spec=CPython3.10.10.final.0-64, system=/usr/bin/python3.10, exe=/home/dharding/code/opensource/tox/.tox/dev/bin/python, platform=linux, version='3.10.10 (main, Mar 25 2023, 18:12:59) [GCC 12.2.1 20230121]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:65]
testenv: 351 I create virtual environment via CPython3Posix(dest=/home/dharding/code/tox_test/.tox/testenv, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:48]
testenv: 352 D create folder /home/dharding/code/tox_test/.tox/testenv/bin [virtualenv/util/path/_sync.py:9]
testenv: 352 D create folder /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages [virtualenv/util/path/_sync.py:9]
testenv: 352 D write /home/dharding/code/tox_test/.tox/testenv/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
testenv: 352 D home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
testenv: 352 D implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
testenv: 352 D version_info = 3.10.10.final.0 [virtualenv/create/pyenv_cfg.py:34]
testenv: 352 D virtualenv = 20.21.0 [virtualenv/create/pyenv_cfg.py:34]
testenv: 352 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
testenv: 352 D base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
testenv: 352 D base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
testenv: 352 D base-executable = /usr/bin/python3.10 [virtualenv/create/pyenv_cfg.py:34]
testenv: 352 D symlink /usr/bin/python3.10 to /home/dharding/code/tox_test/.tox/testenv/bin/python [virtualenv/util/path/_sync.py:28]
testenv: 352 D create virtualenv import hook file /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:89]
testenv: 352 D create /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:92]
testenv: 353 D ============================== target debug ============================== [virtualenv/run/session.py:50]
testenv: 353 D debug via /home/dharding/code/tox_test/.tox/testenv/bin/python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:193]
testenv: 353 D {
"sys": {
"executable": "/home/dharding/code/tox_test/.tox/testenv/bin/python",
"_base_executable": "/home/dharding/code/tox_test/.tox/testenv/bin/python",
"prefix": "/home/dharding/code/tox_test/.tox/testenv",
"base_prefix": "/usr",
"real_prefix": null,
"exec_prefix": "/home/dharding/code/tox_test/.tox/testenv",
"base_exec_prefix": "/usr",
"path": [
"/usr/lib/python310.zip",
"/usr/lib/python3.10",
"/usr/lib/python3.10/lib-dynload",
"/home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages"
],
"meta_path": [
"<class '_virtualenv._Finder'>",
"<class '_frozen_importlib.BuiltinImporter'>",
"<class '_frozen_importlib.FrozenImporter'>",
"<class '_frozen_importlib_external.PathFinder'>"
],
"fs_encoding": "utf-8",
"io_encoding": "utf-8"
},
"version": "3.10.10 (main, Mar 25 2023, 18:12:59) [GCC 12.2.1 20230121]",
"makefile_filename": "/usr/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile",
"os": "<module 'os' from '/usr/lib/python3.10/os.py'>",
"site": "<module 'site' from '/usr/lib/python3.10/site.py'>",
"datetime": "<module 'datetime' from '/usr/lib/python3.10/datetime.py'>",
"math": "<module 'math' from '/usr/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so'>",
"json": "<module 'json' from '/usr/lib/python3.10/json/__init__.py'>"
} [virtualenv/run/session.py:51]
testenv: 370 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/dharding/.local/share/virtualenv) [virtualenv/run/session.py:55]
testenv: 372 D got embed update of distribution setuptools from /home/dharding/.local/share/virtualenv/wheel/3.10/embed/3/setuptools.json [virtualenv/app_data/via_disk_folder.py:129]
testenv: 373 D using periodically updated wheel /home/dharding/.local/share/virtualenv/wheel/house/setuptools-67.5.1-py3-none-any.whl [virtualenv/seed/wheels/periodic_update.py:41]
testenv: 374 D got embed update of distribution wheel from /home/dharding/.local/share/virtualenv/wheel/3.10/embed/3/wheel.json [virtualenv/app_data/via_disk_folder.py:129]
testenv: 374 D got embed update of distribution pip from /home/dharding/.local/share/virtualenv/wheel/3.10/embed/3/pip.json [virtualenv/app_data/via_disk_folder.py:129]
testenv: 375 D install setuptools from wheel /home/dharding/.local/share/virtualenv/wheel/house/setuptools-67.5.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
testenv: 375 D install wheel from wheel /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/wheel-0.38.4-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
testenv: 375 D install pip from wheel /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/pip-23.0.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
testenv: 376 D copy /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.virtualenv to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/wheel-0.38.4.virtualenv [virtualenv/util/path/_sync.py:36]
testenv: 376 D copy /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/distutils-precedence.pth to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:36]
testenv: 376 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.dist-info to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/wheel-0.38.4.dist-info [virtualenv/util/path/_sync.py:36]
testenv: 376 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/_distutils_hack to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:36]
testenv: 377 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/pip [virtualenv/util/path/_sync.py:36]
testenv: 377 D copy /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/setuptools-67.5.1.virtualenv to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/setuptools-67.5.1.virtualenv [virtualenv/util/path/_sync.py:36]
testenv: 378 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/setuptools to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/setuptools [virtualenv/util/path/_sync.py:36]
testenv: 379 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/wheel [virtualenv/util/path/_sync.py:36]
testenv: 386 D generated console scripts wheel-3.10 wheel3.10 wheel3 wheel [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
testenv: 409 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/pkg_resources to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/pkg_resources [virtualenv/util/path/_sync.py:36]
testenv: 415 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/setuptools-67.5.1.dist-info to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/setuptools-67.5.1.dist-info [virtualenv/util/path/_sync.py:36]
testenv: 417 D generated console scripts [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
testenv: 432 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip-23.0.1.dist-info to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/pip-23.0.1.dist-info [virtualenv/util/path/_sync.py:36]
testenv: 432 D copy /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip-23.0.1.virtualenv to /home/dharding/code/tox_test/.tox/testenv/lib/python3.10/site-packages/pip-23.0.1.virtualenv [virtualenv/util/path/_sync.py:36]
testenv: 433 D generated console scripts pip3.10 pip3 pip-3.10 pip [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
testenv: 433 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:61]
testenv: 434 D write /home/dharding/code/tox_test/.tox/testenv/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
testenv: 434 D home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
testenv: 434 D implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
testenv: 434 D version_info = 3.10.10.final.0 [virtualenv/create/pyenv_cfg.py:34]
testenv: 434 D virtualenv = 20.21.0 [virtualenv/create/pyenv_cfg.py:34]
testenv: 434 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
testenv: 434 D base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
testenv: 434 D base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
testenv: 434 D base-executable = /usr/bin/python3.10 [virtualenv/create/pyenv_cfg.py:34]
.pkg: 445 I find interpreter for spec PythonSpec(path=/home/dharding/code/opensource/tox/.tox/dev/bin/python) [virtualenv/discovery/builtin.py:56]
.pkg: 445 D discover exe from cache /usr - exact False: PythonInfo({'architecture': 64, 'base_exec_prefix': '/usr', 'base_prefix': '/usr', 'distutils_install': {'data': '', 'headers': 'include/python3.10/UNKNOWN', 'platlib': 'lib/python3.10/site-packages', 'purelib': 'lib/python3.10/site-packages', 'scripts': 'bin'}, 'exec_prefix': '/usr', 'executable': '/home/dharding/code/opensource/tox/.tox/dev/bin/python', 'file_system_encoding': 'utf-8', 'has_venv': True, 'implementation': 'CPython', 'max_size': 9223372036854775807, 'original_executable': '/usr/bin/python3.10', 'os': 'posix', 'path': ['/home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/virtualenv/discovery', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/usr/lib/python3.10/site-packages'], 'platform': 'linux', 'prefix': '/usr', 'real_prefix': None, 'stdout_encoding': 'utf-8', 'sysconfig': {'makefile_filename': '/usr/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile'}, 'sysconfig_paths': {'data': '{base}', 'include': '{installed_base}/include/python{py_version_short}{abiflags}', 'platlib': '{platbase}/{platlibdir}/python{py_version_short}/site-packages', 'platstdlib': '{platbase}/{platlibdir}/python{py_version_short}', 'purelib': '{base}/lib/python{py_version_short}/site-packages', 'scripts': '{base}/bin', 'stdlib': '{installed_base}/{platlibdir}/python{py_version_short}'}, 'sysconfig_scheme': None, 'sysconfig_vars': {'PYTHONFRAMEWORK': '', 'abiflags': '', 'base': '/usr', 'installed_base': '/usr', 'platbase': '/usr', 'platlibdir': 'lib', 'py_version_short': '3.10'}, 'system_executable': '/usr/bin/python3.10', 'system_stdlib': '/usr/lib/python3.10', 'system_stdlib_platform': '/usr/lib/python3.10', 'version': '3.10.10 (main, Mar 25 2023, 18:12:59) [GCC 12.2.1 20230121]', 'version_info': VersionInfo(major=3, minor=10, micro=10, releaselevel='final', serial=0), 'version_nodot': '310'}) [virtualenv/discovery/py_info.py:435]
.pkg: 446 I proposed PythonInfo(spec=CPython3.10.10.final.0-64, system=/usr/bin/python3.10, exe=/home/dharding/code/opensource/tox/.tox/dev/bin/python, platform=linux, version='3.10.10 (main, Mar 25 2023, 18:12:59) [GCC 12.2.1 20230121]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:63]
.pkg: 446 D accepted PythonInfo(spec=CPython3.10.10.final.0-64, system=/usr/bin/python3.10, exe=/home/dharding/code/opensource/tox/.tox/dev/bin/python, platform=linux, version='3.10.10 (main, Mar 25 2023, 18:12:59) [GCC 12.2.1 20230121]', encoding_fs_io=utf-8-utf-8) [virtualenv/discovery/builtin.py:65]
.pkg: 446 I create virtual environment via CPython3Posix(dest=/home/dharding/code/tox_test/.tox/.pkg, clear=False, no_vcs_ignore=False, global=False) [virtualenv/run/session.py:48]
.pkg: 447 D create folder /home/dharding/code/tox_test/.tox/.pkg/bin [virtualenv/util/path/_sync.py:9]
.pkg: 447 D create folder /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages [virtualenv/util/path/_sync.py:9]
.pkg: 447 D write /home/dharding/code/tox_test/.tox/.pkg/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
.pkg: 447 D home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
.pkg: 447 D implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
.pkg: 447 D version_info = 3.10.10.final.0 [virtualenv/create/pyenv_cfg.py:34]
.pkg: 447 D virtualenv = 20.21.0 [virtualenv/create/pyenv_cfg.py:34]
.pkg: 447 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
.pkg: 447 D base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
.pkg: 447 D base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
.pkg: 447 D base-executable = /usr/bin/python3.10 [virtualenv/create/pyenv_cfg.py:34]
.pkg: 447 D symlink /usr/bin/python3.10 to /home/dharding/code/tox_test/.tox/.pkg/bin/python [virtualenv/util/path/_sync.py:28]
.pkg: 447 D create virtualenv import hook file /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/_virtualenv.pth [virtualenv/create/via_global_ref/api.py:89]
.pkg: 447 D create /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/_virtualenv.py [virtualenv/create/via_global_ref/api.py:92]
.pkg: 447 D ============================== target debug ============================== [virtualenv/run/session.py:50]
.pkg: 448 D debug via /home/dharding/code/tox_test/.tox/.pkg/bin/python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/virtualenv/create/debug.py [virtualenv/create/creator.py:193]
.pkg: 447 D {
"sys": {
"executable": "/home/dharding/code/tox_test/.tox/.pkg/bin/python",
"_base_executable": "/home/dharding/code/tox_test/.tox/.pkg/bin/python",
"prefix": "/home/dharding/code/tox_test/.tox/.pkg",
"base_prefix": "/usr",
"real_prefix": null,
"exec_prefix": "/home/dharding/code/tox_test/.tox/.pkg",
"base_exec_prefix": "/usr",
"path": [
"/usr/lib/python310.zip",
"/usr/lib/python3.10",
"/usr/lib/python3.10/lib-dynload",
"/home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages"
],
"meta_path": [
"<class '_virtualenv._Finder'>",
"<class '_frozen_importlib.BuiltinImporter'>",
"<class '_frozen_importlib.FrozenImporter'>",
"<class '_frozen_importlib_external.PathFinder'>"
],
"fs_encoding": "utf-8",
"io_encoding": "utf-8"
},
"version": "3.10.10 (main, Mar 25 2023, 18:12:59) [GCC 12.2.1 20230121]",
"makefile_filename": "/usr/lib/python3.10/config-3.10-x86_64-linux-gnu/Makefile",
"os": "<module 'os' from '/usr/lib/python3.10/os.py'>",
"site": "<module 'site' from '/usr/lib/python3.10/site.py'>",
"datetime": "<module 'datetime' from '/usr/lib/python3.10/datetime.py'>",
"math": "<module 'math' from '/usr/lib/python3.10/lib-dynload/math.cpython-310-x86_64-linux-gnu.so'>",
"json": "<module 'json' from '/usr/lib/python3.10/json/__init__.py'>"
} [virtualenv/run/session.py:51]
.pkg: 465 I add seed packages via FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=/home/dharding/.local/share/virtualenv) [virtualenv/run/session.py:55]
.pkg: 466 D got embed update of distribution setuptools from /home/dharding/.local/share/virtualenv/wheel/3.10/embed/3/setuptools.json [virtualenv/app_data/via_disk_folder.py:129]
.pkg: 467 D using periodically updated wheel /home/dharding/.local/share/virtualenv/wheel/house/setuptools-67.5.1-py3-none-any.whl [virtualenv/seed/wheels/periodic_update.py:41]
.pkg: 468 D got embed update of distribution pip from /home/dharding/.local/share/virtualenv/wheel/3.10/embed/3/pip.json [virtualenv/app_data/via_disk_folder.py:129]
.pkg: 468 D got embed update of distribution wheel from /home/dharding/.local/share/virtualenv/wheel/3.10/embed/3/wheel.json [virtualenv/app_data/via_disk_folder.py:129]
.pkg: 468 D install setuptools from wheel /home/dharding/.local/share/virtualenv/wheel/house/setuptools-67.5.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
.pkg: 468 D install pip from wheel /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/pip-23.0.1-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
.pkg: 469 D install wheel from wheel /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/virtualenv/seed/wheels/embed/wheel-0.38.4-py3-none-any.whl via CopyPipInstall [virtualenv/seed/embed/via_app_data/via_app_data.py:47]
.pkg: 469 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/pip [virtualenv/util/path/_sync.py:36]
.pkg: 470 D copy /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/distutils-precedence.pth to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/distutils-precedence.pth [virtualenv/util/path/_sync.py:36]
.pkg: 470 D copy /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.virtualenv to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/wheel-0.38.4.virtualenv [virtualenv/util/path/_sync.py:36]
.pkg: 470 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/_distutils_hack to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/_distutils_hack [virtualenv/util/path/_sync.py:36]
.pkg: 471 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel-0.38.4.dist-info to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/wheel-0.38.4.dist-info [virtualenv/util/path/_sync.py:36]
.pkg: 471 D copy /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/setuptools-67.5.1.virtualenv to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/setuptools-67.5.1.virtualenv [virtualenv/util/path/_sync.py:36]
.pkg: 472 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/setuptools to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/setuptools [virtualenv/util/path/_sync.py:36]
.pkg: 473 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/wheel-0.38.4-py3-none-any/wheel to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/wheel [virtualenv/util/path/_sync.py:36]
.pkg: 480 D generated console scripts wheel-3.10 wheel3.10 wheel3 wheel [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
.pkg: 503 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/pkg_resources to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/pkg_resources [virtualenv/util/path/_sync.py:36]
.pkg: 510 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/setuptools-67.5.1-py3-none-any/setuptools-67.5.1.dist-info to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/setuptools-67.5.1.dist-info [virtualenv/util/path/_sync.py:36]
.pkg: 511 D generated console scripts [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
.pkg: 525 D copy directory /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip-23.0.1.dist-info to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/pip-23.0.1.dist-info [virtualenv/util/path/_sync.py:36]
.pkg: 526 D copy /home/dharding/.local/share/virtualenv/wheel/3.10/image/1/CopyPipInstall/pip-23.0.1-py3-none-any/pip-23.0.1.virtualenv to /home/dharding/code/tox_test/.tox/.pkg/lib/python3.10/site-packages/pip-23.0.1.virtualenv [virtualenv/util/path/_sync.py:36]
.pkg: 526 D generated console scripts pip3 pip-3.10 pip pip3.10 [virtualenv/seed/embed/via_app_data/pip_install/base.py:41]
.pkg: 526 I add activators for Bash, CShell, Fish, Nushell, PowerShell, Python [virtualenv/run/session.py:61]
.pkg: 527 D write /home/dharding/code/tox_test/.tox/.pkg/pyvenv.cfg [virtualenv/create/pyenv_cfg.py:30]
.pkg: 527 D home = /usr/bin [virtualenv/create/pyenv_cfg.py:34]
.pkg: 527 D implementation = CPython [virtualenv/create/pyenv_cfg.py:34]
.pkg: 527 D version_info = 3.10.10.final.0 [virtualenv/create/pyenv_cfg.py:34]
.pkg: 527 D virtualenv = 20.21.0 [virtualenv/create/pyenv_cfg.py:34]
.pkg: 527 D include-system-site-packages = false [virtualenv/create/pyenv_cfg.py:34]
.pkg: 528 D base-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
.pkg: 528 D base-exec-prefix = /usr [virtualenv/create/pyenv_cfg.py:34]
.pkg: 528 D base-executable = /usr/bin/python3.10 [virtualenv/create/pyenv_cfg.py:34]
.pkg: 528 W install_requires> python -I -m pip install 'setuptools>=40.8.0' wheel [tox/tox_env/api.py:428]
Requirement already satisfied: setuptools>=40.8.0 in ./.tox/.pkg/lib/python3.10/site-packages (67.5.1)
Requirement already satisfied: wheel in ./.tox/.pkg/lib/python3.10/site-packages (0.38.4)
.pkg: 1241 I exit 0 (0.71 seconds) /home/dharding/code/tox_test> python -I -m pip install 'setuptools>=40.8.0' wheel pid=335223 [tox/execute/api.py:275]
.pkg: 1250 W _optional_hooks> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ [tox/tox_env/api.py:428]
Backend: run command _optional_hooks with args {}
Backend: Wrote response {'return': {'get_requires_for_build_sdist': True, 'prepare_metadata_for_build_wheel': True, 'get_requires_for_build_wheel': True, 'build_editable': True, 'get_requires_for_build_editable': True, 'prepare_metadata_for_build_editable': True}} to /tmp/pep517__optional_hooks-8p38cl4r.json
.pkg: 1394 I exit None (0.14 seconds) /home/dharding/code/tox_test> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ pid=335240 [tox/execute/api.py:275]
.pkg: 1395 W get_requires_for_build_sdist> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ [tox/tox_env/api.py:428]
Backend: run command get_requires_for_build_sdist with args {'config_settings': None}
running egg_info
writing tox_test.egg-info/PKG-INFO
writing dependency_links to tox_test.egg-info/dependency_links.txt
writing top-level names to tox_test.egg-info/top_level.txt
reading manifest file 'tox_test.egg-info/SOURCES.txt'
writing manifest file 'tox_test.egg-info/SOURCES.txt'
Backend: Wrote response {'return': []} to /tmp/pep517_get_requires_for_build_sdist-nqy3wvr1.json
.pkg: 1417 I exit None (0.02 seconds) /home/dharding/code/tox_test> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ pid=335240 [tox/execute/api.py:275]
.pkg: 1418 W prepare_metadata_for_build_wheel> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ [tox/tox_env/api.py:428]
Backend: run command prepare_metadata_for_build_wheel with args {'metadata_directory': '/home/dharding/code/tox_test/.tox/.pkg/.meta', 'config_settings': {'--build-option': []}}
running dist_info
creating /home/dharding/code/tox_test/.tox/.pkg/.meta/tox_test.egg-info
writing /home/dharding/code/tox_test/.tox/.pkg/.meta/tox_test.egg-info/PKG-INFO
writing dependency_links to /home/dharding/code/tox_test/.tox/.pkg/.meta/tox_test.egg-info/dependency_links.txt
writing top-level names to /home/dharding/code/tox_test/.tox/.pkg/.meta/tox_test.egg-info/top_level.txt
writing manifest file '/home/dharding/code/tox_test/.tox/.pkg/.meta/tox_test.egg-info/SOURCES.txt'
reading manifest file '/home/dharding/code/tox_test/.tox/.pkg/.meta/tox_test.egg-info/SOURCES.txt'
writing manifest file '/home/dharding/code/tox_test/.tox/.pkg/.meta/tox_test.egg-info/SOURCES.txt'
creating '/home/dharding/code/tox_test/.tox/.pkg/.meta/tox_test-0.0.0.dist-info'
Backend: Wrote response {'return': 'tox_test-0.0.0.dist-info'} to /tmp/pep517_prepare_metadata_for_build_wheel-x1dyybxp.json
.pkg: 1493 I exit None (0.07 seconds) /home/dharding/code/tox_test> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ pid=335240 [tox/execute/api.py:275]
.pkg: 1494 W build_sdist> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ [tox/tox_env/api.py:428]
Backend: run command build_sdist with args {'sdist_directory': '/home/dharding/code/tox_test/.tox/.pkg/dist', 'config_settings': None}
running sdist
running egg_info
writing tox_test.egg-info/PKG-INFO
writing dependency_links to tox_test.egg-info/dependency_links.txt
writing top-level names to tox_test.egg-info/top_level.txt
reading manifest file 'tox_test.egg-info/SOURCES.txt'
writing manifest file 'tox_test.egg-info/SOURCES.txt'
running check
creating tox_test-0.0.0
creating tox_test-0.0.0/tox_test.egg-info
copying files to tox_test-0.0.0...
copying README -> tox_test-0.0.0
copying setup.py -> tox_test-0.0.0
copying tox_test.egg-info/PKG-INFO -> tox_test-0.0.0/tox_test.egg-info
copying tox_test.egg-info/SOURCES.txt -> tox_test-0.0.0/tox_test.egg-info
copying tox_test.egg-info/dependency_links.txt -> tox_test-0.0.0/tox_test.egg-info
copying tox_test.egg-info/top_level.txt -> tox_test-0.0.0/tox_test.egg-info
Writing tox_test-0.0.0/setup.cfg
Creating tar archive
removing 'tox_test-0.0.0' (and everything under it)
Backend: Wrote response {'return': 'tox_test-0.0.0.tar.gz'} to /tmp/pep517_build_sdist-lr1o6egx.json
.pkg: 1509 I exit None (0.02 seconds) /home/dharding/code/tox_test> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ pid=335240 [tox/execute/api.py:275]
.pkg: 1510 D package .tmp/package/14/tox_test-0.0.0.tar.gz links to .pkg/dist/tox_test-0.0.0.tar.gz (/home/dharding/code/tox_test/.tox) [tox/util/file_view.py:36]
testenv: 1510 W install_package> python -I -m pip install --force-reinstall --no-deps /home/dharding/code/tox_test/.tox/.tmp/package/14/tox_test-0.0.0.tar.gz [tox/tox_env/api.py:428]
Processing ./.tox/.tmp/package/14/tox_test-0.0.0.tar.gz
Preparing metadata (setup.py): started
Preparing metadata (setup.py): finished with status 'done'
Building wheels for collected packages: tox-test
Building wheel for tox-test (setup.py): started
Building wheel for tox-test (setup.py): finished with status 'done'
Created wheel for tox-test: filename=tox_test-0.0.0-py3-none-any.whl size=936 sha256=4898c208cd2bab225a2fc7533086be91487e6dedafeaf0d694e2238fc7936249
Stored in directory: /home/dharding/.cache/pip/wheels/18/8d/22/ffc3acf6c491acdf99a09c8acb1a81a7890f26f2d3b69dd6fd
Successfully built tox-test
Installing collected packages: tox-test
Successfully installed tox-test-0.0.0
testenv: 2616 I exit 0 (1.11 seconds) /home/dharding/code/tox_test> python -I -m pip install --force-reinstall --no-deps /home/dharding/code/tox_test/.tox/.tmp/package/14/tox_test-0.0.0.tar.gz pid=335288 [tox/execute/api.py:275]
testenv: 2617 W commands[0]> python -c 'import os; import sys; sys.exit(not '"'"'BAR'"'"' in os.environ)' [tox/tox_env/api.py:428]
testenv: 2626 C exit 1 (0.01 seconds) /home/dharding/code/tox_test> python -c 'import os; import sys; sys.exit(not '"'"'BAR'"'"' in os.environ)' pid=335316 [tox/execute/api.py:275]
.pkg: 2626 W _exit> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ [tox/tox_env/api.py:428]
Backend: run command _exit with args {}
Backend: Wrote response {'return': 0} to /tmp/pep517__exit-378ndfd1.json
.pkg: 2628 I exit None (0.00 seconds) /home/dharding/code/tox_test> python /home/dharding/code/opensource/tox/.tox/dev/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__ pid=335240 [tox/execute/api.py:275]
.pkg: 2641 D delete package /home/dharding/code/tox_test/.tox/.tmp/package/14/tox_test-0.0.0.tar.gz [tox/tox_env/python/virtual_env/package/pyproject.py:179]
testenv: FAIL code 1 (2.37=setup[2.36]+cmd[0.01] seconds)
evaluation failed :( (2.54 seconds)
Minimal example
If possible, provide a minimal reproducer for the issue:
The following tox.ini works on tox 3.x (the BAR environment variable is present as expected) but fails on tox 4.x (the BAR environment variable should be inherited from [testenv:base]set_env but is absent):
[tox]
envlist =
main
[testenv:base]
set_env =
BAR = 1
[testenv:main]
set_env =
{[testenv:base]set_env}
FOO = {env:FOO2:foo}
commands = python -c "import os; import sys; sys.exit(not 'BAR' in os.environ)"
The issue is that SetEnv.__iter__() gets called recursively, which effectively causes some of the keys to be lost. The first time it is called via VirtualEnv.session(), which ends up invoking ToxEnv.environment_variables(). At this point, the SetEnv._raw dict has one key, FOO, and SetEnv._needs_replacement is ['{[testenv:base]set_env}']. ToxEnv.environment_variables() calls the SetEnv.load() method on each key that SetEnv.__iter__() yields, so once SetEnv.__iter__() yields FOO (as a result of this line), ToxEnv.environment_variables() calls set_env.load('FOO') (see here). In the process of resolving the value for FOO, the replace_env() function ends up being called. This in turn calls SetEnv.__contains__() which in turn calls SetEnv.__iter__() a second time. Because the key being tested (in this case FOO2) is not present, the second call to SetEnv.__iter__() will iterate over all the keys, including expanding the entries from the self._needs_replacement attribute. Those keys will be stored in the self._raw attribute, but since the the first call to SetEnv.__iter__() has already made a copy of the self._raw attribute to protect against concurrent mutation, it cannot yield them. Furthermore, once it is done yielding from its copy of the self._raw keys, the first call finds that self._needs_replacement is empty, because the second call to SetEnv.__iter__() has already processed all of those entries. Thus the first SetEnv.__iter__() call never yields the keys from self._needs_replacement which were expanded by the second .__iter__() call, and from the perspective of the ToxEnv.environment_variables() call, they are lost.
Below is an annotated call stack from when the problem occurs:
File "/usr/lib/python3.10/threading.py", line 973, in _bootstrap
self._bootstrap_inner()
File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
self.run()
File "/usr/lib/python3.10/threading.py", line 953, in run
self._target(*self._args, **self._kwargs)
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 83, in _worker
work_item.run()
File "/usr/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/dharding/code/opensource/tox/src/tox/session/cmd/run/common.py", line 304, in _run
return run_one(
File "/home/dharding/code/opensource/tox/src/tox/session/cmd/run/single.py", line 34, in run_one
skipped, code, outcomes = _evaluate(tox_env, no_test)
File "/home/dharding/code/opensource/tox/src/tox/session/cmd/run/single.py", line 45, in _evaluate
tox_env.setup()
File "/home/dharding/code/opensource/tox/src/tox/tox_env/api.py", line 249, in setup
self._setup_env()
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/runner.py", line 106, in _setup_env
super()._setup_env()
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/api.py", line 198, in _setup_env
self.ensure_python_env()
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/api.py", line 202, in ensure_python_env
conf = self.python_cache()
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/virtual_env/api.py", line 77, in python_cache
base = super().python_cache()
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/api.py", line 240, in python_cache
"version_info": list(self.base_python.version_info),
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/api.py", line 250, in base_python
self._base_python = self._get_python(base_pythons)
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/virtual_env/api.py", line 134, in _get_python
interpreter = self.creator.interpreter
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/virtual_env/api.py", line 126, in creator
return self.session.creator
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/virtual_env/api.py", line 106, in session
env = self.virtualenv_env_vars()
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/virtual_env/api.py", line 111, in virtualenv_env_vars
env = self.environment_variables.copy()
File "/home/dharding/code/opensource/tox/src/tox/tox_env/python/virtual_env/api.py", line 166, in environment_variables
environment_variables = super().environment_variables
File "/home/dharding/code/opensource/tox/src/tox/tox_env/runner.py", line 191, in environment_variables
environment_variables = super().environment_variables
File "/home/dharding/code/opensource/tox/src/tox/tox_env/api.py", line 341, in environment_variables <-- The first loop over SetEnv.__iter__() is happening here.
result[key] = set_env.load(key)
File "/home/dharding/code/opensource/tox/src/tox/config/set_env.py", line 78, in load
result = self._replacer(raw, args) # apply any replace options
File "/home/dharding/code/opensource/tox/src/tox/config/loader/ini/__init__.py", line 77, in replacer
replaced = replace(conf, self, raw_, args_) # do replacements
File "/home/dharding/code/opensource/tox/src/tox/config/loader/ini/replace.py", line 57, in replace
return Replacer(conf, loader, conf_args=args, depth=depth).join(find_replace_expr(value))
File "/home/dharding/code/opensource/tox/src/tox/config/loader/ini/replace.py", line 185, in join
return "".join(self(value))
File "/home/dharding/code/opensource/tox/src/tox/config/loader/ini/replace.py", line 182, in __call__
return [self._replace_match(me) if isinstance(me, MatchExpression) else str(me) for me in value]
File "/home/dharding/code/opensource/tox/src/tox/config/loader/ini/replace.py", line 182, in <listcomp>
return [self._replace_match(me) if isinstance(me, MatchExpression) else str(me) for me in value]
File "/home/dharding/code/opensource/tox/src/tox/config/loader/ini/replace.py", line 197, in _replace_match
replace_value = replace_env(self.conf, args, conf_args)
File "/home/dharding/code/opensource/tox/src/tox/config/loader/ini/replace.py", line 331, in replace_env
if key in set_env:
File "/home/dharding/code/opensource/tox/src/tox/config/set_env.py", line 86, in __contains__
return isinstance(item, str) and item in self.__iter__() <-- The second call to SetEnv.__iter__() happens here
So, question which would guide a potential PR: I've observed another behavior discrepancy between tox 3 and tox 4, and am not sure if I should try to restore the tox 3 behavior or maintain the behavior in tox 4 (I could not find anything in either the tox 3 or tox 4 documentation that specifies the expected behavior).
In tox 3, the order of section substitution relative to explicit keys is significant:
[tox]
envlist =
main
[testenv:base]
setenv =
FOO = 2
[testenv:main]
setenv =
FOO = 3
{[testenv:base]setenv}
will set the environment variable FOO to 2 (the section substitution will take precedence over the explicit key that came before it), while
[tox]
envlist =
main
[testenv:base]
setenv =
FOO = 2
[testenv:main]
setenv =
{[testenv:base]setenv}
FOO = 3
will set the environment variable FOO to 3 (the explicit key, coming after the section substitution, will take precedence).
However, in tox 4, for both of the examples, FOO is 3 (section substitution never takes precedence over explicit keys).
Any input regarding what the desired behavior should be is welcome.
This is expected 🤔 though can be convinced that last found key should win over first found 🤷
So another question, if you don't mind: commit 4cf619a95b2ab14632d2fb6e62335d792da75c37 introduced what looks to me like a level of indirection using concurrent.futures.Future as well as a new .build() method of Loader / IniLoader. I'm not sure of the purpose of this mechanism, or what problem it is trying to solve. Is it possible to shed any light on this?
The core idea is to make configuration definition and manifestation lazy. So tox is now faster because it only does work when it must.
I had to put this issue on the back burner due to other responsibilities, but today I had a bit of time to look at it again. So it looks like tox is making use of futures in tox.session.cmd.run.common._queue_and_wait(), which makes sense to me. However, the usage in tox.config.loader.api I'm still not sure about. Loader.load() creates a Future() manually which the Python docs say should not be done normally, and no executor is involved in that code. Was this some preparatory work that was never finished? As far as I understand this bit of code, it isn't actually doing anything in a lazy manner.
I also faced the same issue.
One note, this only affects the execution environment and not the {env:} substitution.
The substitution done inside tox file works as expected, but the environment of commands is not set properly.
[tox]
envlist = py3,py3_working
[testenv]
skip_install = true
setenv =
ROOTDIR = {toxinidir}
commands =
python -c 'print("ROOTDIR={env:ROOTDIR}")'
python -c 'import os; print(os.environ["ROOTDIR"])'
[testenv:py3]
setenv =
{[testenv]setenv}
EXTRA_VAR = --rootdir={env:TOX_ENV_NAME}
[testenv:py3_working]
setenv =
{[testenv]setenv}
EXTRA_VAR = --rootdir={toxinidir}
tox
py3: commands[0]> python -c 'print("ROOTDIR=/tmp/check_tox_env")'
ROOTDIR=/tmp/check_tox_env
py3: commands[1]> python -c 'import os; print(os.environ["ROOTDIR"])'
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/usr/lib/python3.8/os.py", line 675, in __getitem__
raise KeyError(key) from None
KeyError: 'ROOTDIR'
py3: exit 1 (0.02 seconds) /tmp/check_tox_env> python -c 'import os; print(os.environ["ROOTDIR"])' pid=711194
py3: FAIL ✖ in 0.1 seconds
py3_working: commands[0]> python -c 'print("ROOTDIR=/tmp/check_tox_env")'
ROOTDIR=/tmp/check_tox_env
py3_working: commands[1]> python -c 'import os; print(os.environ["ROOTDIR"])'
/tmp/check_tox_env
py3: FAIL code 1 (0.10=setup[0.05]+cmd[0.02,0.02] seconds)
py3_working: OK (0.06=setup[0.01]+cmd[0.02,0.02] seconds)
evaluation failed :( (0.37 seconds)
A workaround is to move all the variables setting that I want exported out of the common environment.
