Cannot invoke tox with cli options provided by plugins that will be provisioned

[masenf]

Issue

1. A plugin uses tox_add_option to define a new CLI flag, --demo-plugin.
2. Plugin is named in tox.ini [tox] requires section
3. Execute tox --demo-plugin

Instead of provisioning a new environment with the plugin and re-executing tox inside, it bails with

tox: error: unrecognized arguments: --demo-plugin
hint: if you tried to pass arguments to a command use -- to separate them from tox ones

Environment

Provide at least:

• OS: ubuntu 20.04 / python3.8
• updated tox unit test

diff --git a/tests/demo_pkg_inline/build.py b/tests/demo_pkg_inline/build.py
index 22e5dda0..84dbca30 100644
--- a/tests/demo_pkg_inline/build.py
+++ b/tests/demo_pkg_inline/build.py
@@ -23,6 +23,7 @@ content = {
     logic: "def do():\n    print('greetings from {}')".format(name),
     plugin: """
         try:
+            from tox.config.cli.parser import ToxParser
             from tox.plugin import impl
             from tox.tox_env.python.virtual_env.runner import VirtualEnvRunner
             from tox.tox_env.register import ToxEnvRegister
@@ -36,6 +37,9 @@ content = {
             @impl
             def tox_register_tox_env(register: ToxEnvRegister) -> None:
                 register.add_run_env(ExampleVirtualEnvRunner)
+            @impl
+            def tox_add_option(parser: ToxParser) -> None:
+                parser.add_argument("--demo-plugin", action="store_true", help="magical flag")
         """,
 }
 metadata_files = {
diff --git a/tests/test_provision.py b/tests/test_provision.py
index 290e48dc..eabb87bc 100644
--- a/tests/test_provision.py
+++ b/tests/test_provision.py
@@ -233,3 +233,21 @@ def test_provision_conf_file(tox_project: ToxProjectCreator, tmp_path: Path, rel
         conf_path = str(project.path / "tox.ini")
     result = project.run("c", "--conf", conf_path, "-e", "py", from_cwd=tmp_path)
     result.assert_success()
+
+
[email protected]()
[email protected]("_pypi_index_self")
+def test_provision_plugin_options(tox_project: ToxProjectCreator, tmp_path: Path) -> None:
+    """Tox should provision before complaining about unrecognized CLI options."""
+    log = tmp_path / "out.log"
+    proj = tox_project(
+        {"tox.ini": f"[tox]\nrequires=demo-pkg-inline"},
+    )
+    prov_msg = (
+        f"ROOT: will run in automatically provisioned tox, host {sys.executable} is missing"
+        f" [requires (has)]: demo-pkg-inline"
+    )
+
+    result_env = proj.run("r", "-e", "py", "--demo-plugin", "--result-json", str(log))
+    result_env.assert_success()
+    assert prov_msg in result_env.out

I'm looking into a fix, since this bears similarity to #2916 #2862; ultimately i want to find a way to "fast track" provision such that it can occur without loading all of the other parsers and the validation that comes with them, just to re-execute tox in a completely new environment (that is much less likely to hit configuration incompatibilities).

[gaborbernat]

@masenf you still plan on this?

[masenf]

Yes thanks for the nudge. I've had a workaround in place at work so we weren't hitting this anymore, but since I have a test case, a true fix in tox should be forthcoming.

I don't have an ETA at this time, but thanks for keeping it on my radar.