Running pytest in a sub-interpreter

[bluetech]

Python 3.14 added a new module concurrent.interpreters which allows creating isolated sub-interpreters with their own GIL. I like the concept and experimented it with it a bit in pytest-xdist/execnet, trying to use sub-interperters instead of processes. I immediately ran into some problems.

But before getting into xdist and such (which share some issues with #13768, particularly around stdio capturing), the most basic thing is just running pytest in a single sub-interpreter:

#!/usr/bin/env python
import sys
from concurrent import interpreters

interp = interpreters.create()
interp.prepare_main(args=tuple(sys.argv[1:]))
interp.exec("""
import pytest
pytest.main(list(args))
""")

Here are the problems this runs into:

• faulthandler cannot be imported in a sub-interpreter. This may be a fundamental issue as faulthandler is a process-wide concept (https://github.com/python/cpython/issues/101509). I am not sure yet. To make this work we may want to disable auto-loading the faulthandler plugin and skip its tests (or run them in a subprocess) when running under a sub-interpreter.

• multiprocessing.Pool used by a couple of our tests testing/acceptance_test.py::TestGeneralUsage::test_config_error and testing/_py/test_local.py::TestLocalPath::test_make_numbered_dir_multiprocess_safe is incompatible https://github.com/python/cpython/issues/140057

• readline imported by test test_libedit_workaround is incompatible