junit-plugin icon indicating copy to clipboard operation
junit-plugin copied to clipboard

Published 20 hours ago • verified publisher icon jenkinsci

Show all failed tests

Open brianjmurrell opened this issue 2 years ago • 5 comments

What feature do you want to see added?

Somewhat along the lines of #492, it would also be useful to list all tests, or at minimum all failed/errored tests. For example:

image

says there are 2 failed tests yet only one failed test result (and no passing tests are even mentioned) is being shown.

Upstream changes

No response

brianjmurrell avatar Feb 15 '23 20:02 brianjmurrell

Failing tests are likely much more interesting than passing tests. We try to send as much useful data as we can within the character count limit.

Improvements welcome though

timja avatar Feb 16 '23 11:02 timja

Failing tests are likely much more interesting than passing tests.

True. But you totally missed the headline. I am not even seeing all of my failed tests. See the screen shot. it shows failed: 2 but then below is only a single failed result. Where is the other one?

brianjmurrell avatar Feb 16 '23 12:02 brianjmurrell

But you totally missed the headline

I didn't. See the bottom, 'more test results are not shown here', that text is added when the plugin attempted to add more detail but was unable to because it would put it over the character limit.

timja avatar Feb 16 '23 12:02 timja

Oh my goodness. This is seriously disappointing. I really don't see the point of posting anything to GH Checks if you cannot post all of the failed tests and post all of the details of the failed tests.

The grand hope here was to avoid users having to go to a Jenkins instance because (a) it so frequently needs updating due to the constant stream of vulnerabilities that are issued against it (and thus we'd like to keep it walled off of the general Internet) and (b) the UI(s) are simply horribly quirky and bug ridden and all of the bugs about them are simply ignored. Don't even think of trying to use Blue Ocean if you don't have HTTP/2 capability. But I digress, and I do realize that none of that is your fault, and neither are the limitations of character limits which seriously devalue these GH Checks from Jenkins.

Can you expand on the character limit that you are referring to? I would like to investigate that further and see if there is any way to ameliorate it.

brianjmurrell avatar Feb 16 '23 13:02 brianjmurrell

see https://github.com/jenkinsci/junit-plugin/issues/492#issuecomment-1433153764

timja avatar Feb 16 '23 14:02 timja