tox-dev
envbindir behavior changed in tox4
[tox]
skipsdist = true
envlist = py{37,38}
[testenv]
deps = flake8
commands =
py37: {[testenv:py37-flake8]commands}
py38: {[testenv:py38-flake8]commands}
[testenv:py{37,38}-flake8]
commands = {envbindir}/flake8 --version
$ venv/bin/tox --version
3.28.0 imported from /tmp/tmp.5W879ZcLx9/venv/lib/python3.8/site-packages/tox/__init__.py
$ venv/bin/tox
py37 installed: flake8==5.0.4,importlib-metadata==4.2.0,mccabe==0.7.0,pycodestyle==2.9.1,pyflakes==2.5.0,typing_extensions==4.4.0,zipp==3.12.0
py37 run-test-pre: PYTHONHASHSEED='335891262'
py37 run-test: commands[0] | /tmp/tmp.5W879ZcLx9/.tox/py37/bin/flake8 --version
5.0.4 (mccabe: 0.7.0, pycodestyle: 2.9.1, pyflakes: 2.5.0) CPython 3.7.13 on
Linux
py38 installed: flake8==6.0.0,mccabe==0.7.0,pycodestyle==2.10.0,pyflakes==3.0.1
py38 run-test-pre: PYTHONHASHSEED='335891262'
py38 run-test: commands[0] | /tmp/tmp.5W879ZcLx9/.tox/py38/bin/flake8 --version
6.0.0 (mccabe: 0.7.0, pycodestyle: 2.10.0, pyflakes: 3.0.1)
CPython 3.8.10 on Linux
____________________________ summary _____________________________
py37: commands succeeded
py38: commands succeeded
congratulations :)
- Note how
{envbindir}was substituted using the name of the env being run:py38 run-test: commands[0] | /tmp/tmp.5W879ZcLx9/.tox/py38/bin/flake8 --version
$ venv/bin/tox --version
4.4.2 from /tmp/tmp.5W879ZcLx9/venv/lib/python3.8/site-packages/tox/__init__.py
$ venv/bin/tox
py37: commands[0]> .tox/py37-flake8/bin/flake8 --version
py37: exit 2 (0.01 seconds) /tmp/tmp.5W879ZcLx9> .tox/py37-flake8/bin/flake8 --version
py37: FAIL ✖ in 0.1 seconds
py38: commands[0]> .tox/py38-flake8/bin/flake8 --version
py38: exit 2 (0.00 seconds) /tmp/tmp.5W879ZcLx9> .tox/py38-flake8/bin/flake8 --version
py37: FAIL code 2 (0.10=setup[0.09]+cmd[0.01] seconds)
py38: FAIL code 2 (0.02=setup[0.02]+cmd[0.00] seconds)
evaluation failed :( (0.23 seconds)
- Note how the
{envbindir}substitution changed to the name of the env being referenced:py38: commands[0]> .tox/py38-flake8/bin/flake8 --version
Was this intentional?
I played around with this a bit tonight and it's unclear what the desired behavior is.
As it's written in tox 4.0.0+, replacement values from another section are looked up exclusively in the context of that other section (as you've seen). In this code path, the replacements are expanded in a depth first fashion.
It's not clear what is desirable when the command references some value from the local environment and then also gets included from another environment.
- should all of the substitutions be evaluated in the context of the including environment?
- should all of the substitutions be evaluated in the context of the lexical environment the inclusion is defined within?
- some combination of the above?
With the following patch, your repro seems to behave like you expect, and I'm only failing one unit test:
diff --git a/src/tox/config/loader/ini/replace.py b/src/tox/config/loader/ini/replace.py
index 41672780..bb1b79a4 100644
--- a/src/tox/config/loader/ini/replace.py
+++ b/src/tox/config/loader/ini/replace.py
@@ -252,6 +252,10 @@ def replace_reference(conf: Config, loader: IniLoader, value: str, conf_args: Co
try:
if isinstance(src, SectionProxy):
return loader.process_raw(conf, conf_args.env_name, src[key])
+ if settings["env"] is not None:
+ # when grabbing from another section, don't expand in the context of _that_ section
+ for loader in src.loaders:
+ return loader.load_raw(key, conf, conf_args.env_name)
value = src.load(key, conf_args.chain)
as_str, _ = stringify(value)
as_str = as_str.replace("#", r"\#") # escape comment characters as these will be stripped
☝️ this has it's own problems though... for example with the following change to your tox.ini
[testenv:py{37,38}-flake8]
install_command = echo this wont work {packages}
commands = {envbindir}/flake8 --version {install_command}
This issue represents a clear regression from tox 3, and we need a decision on what will actually be supported w.r.t. cross-environment references.
I never understood why people use envbindir. It's not needed, as that's always first on path. Can someone explain?
I never understood why people use envbindir
Likely superfluous in this case, as I think a workaround would be to simply NOT specify {envbindir}.
The bigger problem uncovered by this issue though is a major regression from tox3 in how values included from other sections are interpreted; this issue is reproducible with {envdir}, {envname}, {env_tmp_dir}, and others. It especially causes a problem when trying to create files in env-specific directories.
For example
[tox]
requires = tox == 4.4.3
skipsdist = true
envlist = py, foo
[testenv]
commands =
{[testenv:foo]commands}
[testenv:foo]
commands =
python -c "print('{envname} {envdir} {env_tmp_dir}')"
python -c "from pathlib import Path; (Path('{env_tmp_dir}') / 'foo').write_text('foobarbaz')"
☝️ that doesn't work the first time, because .tox/foo doesn't exist.
If the tox4 behavior is indeed the intended design, then it needs to be documented more accurately on the upgrading to tox4 page.
Going off the patch I posted earlier, if we add the syntax that {[.]blah} refers to the blah key in the current lexicographical definition, then we get some nice properties:
- specifying a section reference (
{[envname]commands}) can always substitute in the raw value from the other section - a section self-reference,
[.], will be replaced by the defining environment name, including factors (escape hatch to retain tox4 behavior) - replaced values are recursively substituted, so any unqualified substitutions (
{envdir}) will always expand in the context of the executing environment, not the defining environment (tox3 behavior)
I personally think the observed behavior in tox 4.4.3 is a bug that should be fixed, to better approximate how this was handled in tox 3.