Unhandled exceptions in `pytest_unconfigure` hooks exit `1` and not `3` - `ExitCode.INTERNAL_ERROR`

[symonk]

Hi, as (briefly) discussed in this discussion, pytest_unconfigure differs from pytest_configure in terms of exiting.

When a pytest_configure(...) implementation raises an unhandled exception; pytest exits 3 - ExitCode.INTERNAL_ERROR; this is outlined below:

# conftest.py
import pytest

@pytest.hookimpl
def pytest_configure(config: pytest.Config):
    raise Exception("Unhandled exception; exit will be 3.")

# test_foo.py

def test_foo():
    assert True

pytest .
echo $?
3

However; the reverse is not true, an unhandled exception in the pytest_unconfigure(...) cases forces pytest to exit 1 instead.

# conftest.py
import pytest

@pytest.hookimpl
def pytest_configure(config: pytest.Config) -> None:
    raise Exception("Unhandled exc; exit will be 3.")


@pytest.hookimpl
def pytest_unconfigure(config: pytest.Config) -> None:
    raise Exception("No internal error here; pytest will exit 1")

# test_foo.py

def test_foo():
    assert True

and a 1 exit code:

    raise Exception("No internal error here; pytest will exit 1")
Exception: No internal error here; pytest will exit 1
(venv)  ✘ sy  ~/PycharmProjects/pythonProject45  echo $?
1

[RonnyPfannschmidt]

https://github.com/pytest-dev/pytest/blob/c326c0449433ff812e1b3b81b2590f0d1d797fde/src/_pytest/main.py#L255-L311

we pretty much dont handle exceptons from that particualr hook

on a sidenote, i also beleive its not expected that it causes issues in the form of exceptions - its too late for errors

[symonk]

I've just started having a look at this one; heres an example test to prove out the case:

def test_unconfigure_exc_handling(pytester: Pytester) -> None:
    pytester.makeconftest("""
    import pytest
    
    @pytest.hookimpl
    def pytest_configure(config: pytest.Config) -> None:
        raise Exception("before")
    
    
    @pytest.hookimpl
    def pytest_unconfigure(config: pytest.Config) -> None:
        raise Exception("after")
        """)
    pytester.makepyfile("""
        def test_foo():
            assert True
            
        def test_bar():
            assert False
    """)
    output = pytester.runpytest_subprocess()
    assert output.ret == ExitCode.TESTS_FAILED

@RonnyPfannschmidt @nicoddemus my question is: what is the best approach here in 'solving' it? do we need to guard the cleanup (pytest_unconfigure and registered callables on the config? There is a lot of guarding on on the setup/runtestloop aspects of things and none here as Ronny mentions, on the 'it's too late for errors' could you elaborate that and how you would expect it to be handled?

Thanks!