ImportError with tox but pytest works when using entry_points plugins.

[tdenewiler]

Issue

I expect tox to succeed when tests pass with pytest. I am seeing all tests pass with pytest but plugin project has unit tests fail when using tox due to ImportErrors.

This might be out of scope for requesting help. I have tried many different approaches to fix my issue when running tox. If you can provide help it would be greatly appreciated. I have not been able to find examples of other projects with our setup: a main Python package in one repository with support for extending functionality via plugins in Python packages in separate repositories.

Environment

Minimal working example Python projects on Ubuntu 22.04 with Python 3.10.12, tox 4.14.2, pytest 8.0.1. I have a full project that runs GitHub actions on Mac, Linux, and Windows where we are seeing the same issue.

Output of running tox

$ tox -e py310
.pkg: _optional_hooks> python /home/thomas/src/tox-pyproject/tox-pyproject/venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: get_requires_for_build_sdist> python /home/thomas/src/tox-pyproject/tox-pyproject/venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg: build_sdist> python /home/thomas/src/tox-pyproject/tox-pyproject/venv/lib/python3.10/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py310: install_package> python -I -m pip install --force-reinstall --no-deps /home/thomas/src/tox-pyproject/tox-pyproject/tox-pyproject-plugins/.tox/.tmp/package/5/tox-pyproject-plugins-0.0.1.tar.gz
py310: commands[0] /home/thomas/src/tox-pyproject/tox-pyproject/tox-pyproject-plugins/.output-py310> pytest --cov=/home/thomas/src/tox-pyproject/tox-pyproject/tox-pyproject-plugins/src/tox_pyproject --cov-report term-missing --doctest-modules --junit-xml=tox-pyproject-py310-junit.xml --junit-prefix=py310 /home/thomas/src/tox-pyproject/tox-pyproject/tox-pyproject-plugins
========================================================================== test session starts ==========================================================================
platform linux -- Python 3.10.12, pytest-8.1.1, pluggy-1.4.0
cachedir: .tox/py310/.pytest_cache
rootdir: /home/thomas/src/tox-pyproject/tox-pyproject/tox-pyproject-plugins
configfile: tox.ini
plugins: cov-5.0.0
collected 0 items / 3 errors                                                                                                                                            

================================================================================ ERRORS =================================================================================
_______________________________________________________ ERROR collecting src/tox_pyproject/plugins/discovery/b.py _______________________________________________________
ImportError while importing test module '/home/thomas/src/tox-pyproject/tox-pyproject/tox-pyproject-plugins/src/tox_pyproject/plugins/discovery/b.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
../src/tox_pyproject/plugins/discovery/b.py:3: in <module>
    from tox_pyproject.config import Config
E   ModuleNotFoundError: No module named 'tox_pyproject.config'
__________________________________________________________________ ERROR collecting tests/b/test_b.py ___________________________________________________________________
ImportError while importing test module '/home/thomas/src/tox-pyproject/tox-pyproject/tox-pyproject-plugins/tests/b/test_b.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
../tests/b/test_b.py:5: in <module>
    from tox_pyproject.plugins.discovery.b import BDiscoveryPlugin
../src/tox_pyproject/plugins/discovery/b.py:3: in <module>
    from tox_pyproject.config import Config
E   ModuleNotFoundError: No module named 'tox_pyproject.config'
__________________________________________________________________ ERROR collecting tests/b/test_b.py ___________________________________________________________________
ImportError while importing test module '/home/thomas/src/tox-pyproject/tox-pyproject/tox-pyproject-plugins/tests/b/test_b.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
../tests/b/test_b.py:5: in <module>
    from tox_pyproject.plugins.discovery.b import BDiscoveryPlugin
../src/tox_pyproject/plugins/discovery/b.py:3: in <module>
    from tox_pyproject.config import Config
E   ModuleNotFoundError: No module named 'tox_pyproject.config'

One other thing I tried is to activate the tox virtual environment then start a Python session. In that Python session I am able to import the modules that cause errors when running tests with tox. The files appear to exist where expected, but something about running tox is different than what I am expecting to happen.

Minimal example

I have a repository at https://github.com/tdenewiler/tox-pyproject with a minimal working example (still fairly large) where the main project is fine, but using plugins in a separate project causes failures with tox testing. The README in the plugin package has details on how to execute, what works, and what doesn't work (https://github.com/tdenewiler/tox-pyproject/tree/main/tox-pyproject-plugins).

The main project we have is at https://github.com/sscpac/statick. I am trying to move away from yapsy for plugins and use a standard library-based implementation for plugin support. The forks where I have ported the code to use entry_points from standard library (and where I am seeing tox testing failures in plugin repos) is at:

• https://github.com/tdenewiler/statick/tree/stdlib-plugins (main)
• https://github.com/tdenewiler/statick-tex/tree/stdlib-plugins (plugins)
• https://github.com/tdenewiler/statick-md/tree/stdlib-plugins (plugins)

Example of failures in GitHub workflows:

https://github.com/tdenewiler/statick-md/actions/runs/8585765290