Pytest 7.4.0 causing skip reason for long test names to disappear in small terminal

[clee2000]

trafficstars

Hi, after updating to pytest-7.4.0, I noticed that skip reasons for long tests names either get cutoff or disappear when running pytest in a small width terminal or piping output to a file.

I'm pretty sure this is due to https://github.com/pytest-dev/pytest/pull/10958/files. Since the test name is longer than the terminal width, the extra whitespace that is added to pad the line gets wrapped and then dropped, causing the removal later to be wrong.

tinyrepro.py:

import pytest

@pytest.mark.skip("Random skip reason")
def test_really_really_really_really_really_really_really_long_test_name():
    assert 1 == 1


@pytest.mark.skip("Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.")
def test_really_really_really_really_really_really_really_long_test_name2():
    assert 1 == 1

Output from pytest tinyrepro.py -vv when my terminal has width 236

=========================================================================================================== test session starts ============================================================================================================
platform darwin -- Python 3.11.3, pytest-7.4.0, pluggy-1.2.0 -- /Users/csl/opt/anaconda3/envs/delete/bin/python
cachedir: .pytest_cache
rootdir: /Users/csl/zzzzzzzz/random-testing
collected 2 items                                                                                                                                                                                                                          

tinyrepro.py::test_really_really_really_really_really_really_really_long_test_name SKIPPED [0.0001s] (Random skip reason)                                                                                                            [ 50%]
tinyrepro.py::test_really_really_really_really_really_really_really_long_test_name2 SKIPPED [0.0001s] (Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.
Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint
occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.)                                                                                                                                     [100%]

============================================================================================================ 2 skipped in 0.01s ============================================================================================================

Output with terminal width 90:

================================== test session starts ===================================
platform darwin -- Python 3.11.3, pytest-7.4.0, pluggy-1.2.0 -- /Users/csl/opt/anaconda3/envs/delete/bin/python
cachedir: .pytest_cache
rootdir: /Users/csl/zzzzzzzz/random-testing
collected 2 items                                                                        

tinyrepro.py::test_really_really_really_really_really_really_really_long_test_name SKIPPED [0.0002s] [ 50%]
tinyrepro.py::test_really_really_really_really_really_really_really_long_test_name2 SKIPPED [0.0001s]ncididunt ut labore et dolore magna aliqua. Ut enim ad minim
veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum
dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident,
sunt in culpa qui officia deserunt mollit anim id est laborum.)                    [100%]

=================================== 2 skipped in 0.00s ===================================

Contents of output.txt from pytest tinyrepro.py -vv > output.txt

============================= test session starts ==============================
platform darwin -- Python 3.11.3, pytest-7.4.0, pluggy-1.2.0 -- /Users/csl/opt/anaconda3/envs/delete/bin/python
cachedir: .pytest_cache
rootdir: /Users/csl/zzzzzzzz/random-testing
collecting ... collected 2 items

tinyrepro.py::test_really_really_really_really_really_really_really_long_test_name SKIPPED [0.0001s] [ 50%]
tinyrepro.py::test_really_really_really_really_really_really_really_long_test_name2 SKIPPED [0.0001s]d tempor incididunt ut labore et dolore
magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco
laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor
in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla
pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa
qui officia deserunt mollit anim id est laborum.)                        [100%]

============================== 2 skipped in 0.00s ==============================

With pytest-7.3.2, terminal width 90, pytest tinyrepro.py -vv looks like

================================== test session starts ===================================
platform darwin -- Python 3.11.3, pytest-7.3.2, pluggy-1.2.0 -- /Users/csl/opt/anaconda3/envs/delete/bin/python
cachedir: .pytest_cache
rootdir: /Users/csl/zzzzzzzz/random-testing
collected 2 items                                                                        

tinyrepro.py::test_really_really_really_really_really_really_really_long_test_name SKIPPED [0.0002s] (Random skip reason) [ 50%]
tinyrepro.py::test_really_really_really_really_really_really_really_long_test_name2 SKIPPED [0.0001s] (Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.) [100%]

=================================== 2 skipped in 0.01s ===================================

Output of pip list:

sh: /opt/homebrew/bin/pip: bad interpreter: /opt/homebrew/opt/python35/bin/python3.5: no such file or directory
Package    Version
---------- -------
iniconfig  2.0.0
packaging  23.1
pip        23.1.2
pluggy     1.2.0
pytest     7.4.0
setuptools 67.8.0
wheel      0.38.4

Pytest version: 7.4.0 OS: macOS Ventura 13.4

[radkujawa]

I also encountered this. This is a big disadvantage, as output of pytest in Jenkins stopped printing skip reason at all. I need this skip reason, because I'm parallelizing tests on many jenkins workers, and skip reason was telling me on which machine the test was actually executed.

[jmichiel]

I have been bitten by this as well. I wrote a GUI for running out test-suite that parses the -vv output. Now all of a sudden, SKIP reasons were missing, or the parser completely lost track, because some lines are split in 2 lines. (see https://github.com/pytest-dev/pytest/blame/afb8d66e42a3449476cf9bf4526705b1e36ff5a5/src/_pytest/terminal.py#L449) I had to revert to pytest 7.3.0. An option to prevent this behaviour would be greatly appreciated!

[The-Compiler]

I wrote a GUI for running out test-suite that parses the -vv output.

You might want to consider using pytest-reportlog or writing a custom plugin to get the output in whatever way you want instead. The terminal output is not stable/parsable and never was intended to be.

[jmichiel]

You might want to consider using pytest-reportlog or writing a custom plugin to get the output in whatever way you want instead. The terminal output is not stable/parsable and never was intended to be.

Thanks for the tip, I'll certainly look into it!

[radkujawa]

pytest-reportlog is definitely not what I am searching for. I want to read jenkins output using my own eyes to be able quickly determine the machine on which the test is executed. Grepping through any kind of report log would be extremely annoying.

Please add an option for unconditionally printing the skip reason.

[RonnyPfannschmidt]

@bluetech any idea for adding this to the output in a manner thats not completely messing it up

[disconnect3d]

Bump. We also got bitten by it in https://github.com/pwndbg/pwndbg/pull/3002#issuecomment-2902112623.

You might want to consider using pytest-reportlog or writing a custom plugin to get the output in whatever way you want instead. The terminal output is not stable/parsable and never was intended to be.

While I understand the terminal output is not stable/parsable isn't it just buggy in the first place? Why does the terminal width make pytest output a truncated text instead of all of it?

Any chance that this gets fixed in the sooner than later future? It has been... some time :).

[disconnect3d]

Since this bug was introduced in Pytest 7.4.0 I looked into the diff between pytest 7.3.2 and 7.4.0: https://github.com/pytest-dev/pytest/compare/7.3.2..7.4.0 and I think that this bug was introduced by https://github.com/pytest-dev/pytest/pull/10958

EDIT: Lol, I didn't read the whole issue in the first place and well, it links the problematic PR, heh :)

Specifically, the truncation happens here: https://github.com/pytest-dev/pytest/blob/85a76b84296263eb6b13b59dd0641ff2f920dae2/src/_pytest/terminal.py#L502

Commenting this line on latest pytest (8.3.5) changes the output so that no skip reason truncation is happening. The comparison of the two outputs can be seen below.

1. Non-changed pytest 8.3.5 with small console width:

2. Pytest 8.3.5 with the abovementioned line commented out and with small console width:

The latter shows proper skipped reason ("Needs review.In the heap.py on the line 972 the condition istrue always. The heap_bug.c file has the function:corrupted_unsorted_chunks()"), even though it is written into a new line.

[gmyers18]

Ran into this today when examining terminal output from GitLab CI. Was able to work around it by setting COLUMNS environment variable to a sufficiently large value. This worked for both bash and Windows cmd environments. In my case COLUMNS=150 was the speed spot for the width.