Add line of crash to new summary

[blueyed]

The new summary uses the line of the test function itself, but the line containing the crash would be more useful often (to jump directly to it).

I think this could still get parsed from crashrepr, no?

I think both line numbers could be displayed in the summary similar to https://github.com/fschulze/pytest-pdb/pull/3/files#diff-30e02d59448c958f16700f2683820a2cR28.

[azmeuk]

What is crashrepr? Google has almost nothing about this.

[blueyed]

Sorry, I've meant reprcrash.

https://github.com/pytest-dev/pytest/blob/db24a3b0fb6803c67718852958de4008dcf69319/_pytest/terminal.py#L448-L455

Used in pytest-sugar here: https://github.com/Frozenball/pytest-sugar/blob/4cf631063ee154b1c7c8bafc52514bf393fb9c68/pytest_sugar.py#L533

[Teemu]

I have personally noticed that location of the actual crash is often meaningless (e.g. inside some third party library when invalid argument is passed)

[blueyed]

IIRC it should be from the actual test always?! Otherwise it would be possible to get to it somehow probably. See https://github.com/fschulze/pytest-pdb/pull/3.

[blueyed]

@Frozenball Assuming that the crash line is from the test, do you agree that it would be more useful? The use case is quickly going to the failed assertion etc in the test, which is currently not possible, even if you have setup your editor to handle test IDs (test_foo.py::test_func), because pytest-sugar also does not use those in the summary. I'd be happy to look into this, but would like to know first if it would be considered.

[blueyed]

To be clear, currently it looks like this:

 tests/test_foo.py ✓✓✓✓✓✓✓✓

――――― test_foo_bar ――――――
tests/test_foo.py:985: in test_foo_bar
    assert m.call_count == 1
E   assert 2 == 1
E    +  …
------ Captured stderr setup ------
…

 tests/test_foo.py ⨯✓✓✓s✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓

――――――― test_foo_baz ―――――
tests/test_foo.py:1064: in test_foo_baz
    …
E   Exception: something_bad

 tests/test_foo.py ⨯✓✓✓✓                                                                                                                            100% ██████████
======== short test summary info =======
FAIL tests/test_foo.py::test_foo_bar
FAIL tests/test_foo.py::test_foo_baz

Results (7.96s):
      32 passed
       2 failed
         - tests/test_foo.py:969 test_foo_bar
         - tests/test_foo.py:998 test_foo_baz
       1 skipped

Since I am using -ra I get the test IDs in the short test summary info, which allows to quickly go to the test itself already (although it requires magic in your editor).

I think -ra would not be necessary at all, and pytest-sugar could display the results as:

Results (7.96s):
      32 passed
       2 failed
         - tests/test_foo.py:985 test_foo::test_foo_bar
         - tests/test_foo.py:1064 test_foo::test_foo_baz
       1 skipped

I.e. it would use the actual line, and also display the full test ID.

[Teemu]

Sounds good.

On Wed, Aug 29, 2018, 17:59 Daniel Hahler [email protected] wrote:

@Frozenball https://github.com/Frozenball Assuming that the crash line is from the test, do you agree that it would be more useful? The use case is quickly going to the failed assertion etc in the test, which is currently not possible, even if you have setup your editor to handle test IDs (test_foo.py::test_func), because pytest-sugar also does not use those in the summary. I'd be happy to look into this, but would like to know first if it would be considered.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Frozenball/pytest-sugar/issues/125#issuecomment-416984901, or mute the thread https://github.com/notifications/unsubscribe-auth/AADQMjI5C7284KfLObRLfjz4ocJ4_vsVks5uVqxigaJpZM4OOiPs .

[Teemu]

In that specific case it's good. The general idea is that we wanna jump to the test since it's most likely that our test code is wrong or we are interested what our test does.

There are edge cases like where you are using helper function for testing and the assert statement fails there. In that case I wouldn't be interested in the generic helper function but rather jumping into the test that calls that function. 👷

On Wed, Aug 29, 2018, 18:13 Daniel Hahler [email protected] wrote:

To be clear, currently it looks like this:

tests/test_foo.py ✓✓✓✓✓✓✓✓

――――― test_foo_bar ―――――― tests/test_foo.py:985: in test_foo_bar assert m.call_count == 1 E assert 2 == 1 E + … ------ Captured stderr setup ------ …

tests/test_foo.py ⨯✓✓✓s✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓

――――――― test_foo_baz ――――― tests/test_foo.py:1064: in test_foo_baz … E Exception: something_bad

tests/test_foo.py ⨯✓✓✓✓ 100% ██████████ ======== short test summary info ======= FAIL tests/test_foo.py::test_foo_bar FAIL tests/test_foo.py::test_foo_baz

Results (7.96s): 32 passed 2 failed - tests/test_foo.py:969 test_foo_bar - tests/test_foo.py:998 test_foo_baz 1 skipped

Since I am using -ra I get the test IDs in the short test summary info, which allows to quickly go to the test itself already (although it requires magic in your editor).

I think -ra would not be necessary at all, and pytest-sugar could display the results as:

Results (7.96s): 32 passed 2 failed - tests/test_foo.py:985 test_foo::test_foo_bar - tests/test_foo.py:1064 test_foo::test_foo_baz 1 skipped

I.e. it would use the actual line, and also display the full test ID.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Frozenball/pytest-sugar/issues/125#issuecomment-416989771, or mute the thread https://github.com/notifications/unsubscribe-auth/AADQMmbUcQuTlEP1T1Q8SGVmBsveibG4ks5uVq-WgaJpZM4OOiPs .

[blueyed]

Yes, that's what I would like to achieve/see also.

[blueyed]

btw: I can highly recommend https://github.com/fschulze/pytest-pdb, which adds a gt command that allows to go the test from within pdb and more.