pytest icon indicating copy to clipboard operation
pytest copied to clipboard

Run pexpect tests on CI by default

Open The-Compiler opened this issue 1 year ago • 0 comments

  • pytester tests that require pexpect are silently skipped if pexpect is unavailable:

https://github.com/pytest-dev/pytest/blob/f373974707f57a0b28d12563e4d03c7cd54c70d9/src/_pytest/pytester.py#L1513

  • We default to not installing pexpect even on CI, only having it as an optional dependency with a -pexpect factor:

https://github.com/pytest-dev/pytest/blob/f373974707f57a0b28d12563e4d03c7cd54c70d9/tox.ini#L79

  • The only CI environment where we include that is a Python 3.8 one:

https://github.com/pytest-dev/pytest/blob/f373974707f57a0b28d12563e4d03c7cd54c70d9/.github/workflows/test.yml#L116-L120

  • Yet, we have 40 (!) selftests that use pexpect - most notably, most of test_debugging.py (38 of 59 skipped)

  • And that caused us to miss #12497 on our own CI

Why is pexpect an optional factor at all? From what I can gather, that was introduced in bd8a2cc18c1898673665023556e73134e78e4d75 because "it doesn't install on Windows anymore" back in 2013.

Nowadays, it seems to be partially available for Windows, though not pexpect.spawn which we use for Pytester.spawn():

https://github.com/pytest-dev/pytest/blob/f373974707f57a0b28d12563e4d03c7cd54c70d9/src/_pytest/pytester.py#L1508-L1522

I think we should either:

  • Figure out if we can change Pytester in a backwards-compatible way to use the cross-platform PopenSpawn instead. Pexpect claims:

PopenSpawn is not a direct replacement for spawn. Many programs only offer interactive behaviour if they detect that they are running in a terminal. When run by PopenSpawn, they may behave differently.

  • Or if not, at least install pexpect unconditionally on Linux (and perhaps macOS) as part of our dev dependencies, using a environment marker (added in pip 6.0 in 2014, so that wasn't an option back when the tox factor was introduced).

The-Compiler avatar Oct 13 '24 11:10 The-Compiler