Failed test names when BeforeAll fails aren't shown

[davidmatson]

Checklist

• [x] Issue has a meaningful title
• [x] I have searched the existing issues. See all issues
• [x] I have tested using the latest version of Pester. See Installation and update guide.

What is the issue?

Failed test aren't shown, even though Pester knows they exist.

Pester v6.0.0-alpha5

Starting discovery in 1 files.
Discovery found 6 tests in 11ms.
Running tests.

Running tests from ''
[-] <ScriptBlock>:C:\some_path\BeforeAll.Tests.ps1:27 failed with:
RuntimeException: bad error
at <ScriptBlock>, C:\some_path\BeforeAll.Tests.ps1:29
Tests completed in 50ms
Tests Passed: 0, Failed: 6, Skipped: 0, Inconclusive: 0, NotRun: 0
Container failed: 1
  - <ScriptBlock>:C:\some_path\BeforeAll.Tests.ps1:27

Expected Behavior

Pester v6.0.0-alpha5

Starting discovery in 1 files.
Discovery found 6 tests in 83ms.
Running tests.

Running tests from ''
[-] <ScriptBlock>:C:\some_path\BeforeAll.Tests.ps1:27 failed with:
RuntimeException: bad error
at <ScriptBlock>, C:\some_path\BeforeAll.Tests.ps1:29

Describing d1
  [-] i1 90ms (0ms|0ms)
  [-] i2 9ms (0ms|0ms)
 Describing d2
   [-] i3 15ms (0ms|0ms)
   [-] i4 11ms (0ms|0ms)

Describing d3
  [-] i5 15ms (0ms|0ms)
  [-] i6 12ms (0ms|0ms)
Tests completed in 412ms
Tests Passed: 0, Failed: 6, Skipped: 0, Inconclusive: 0, NotRun: 0

Steps To Reproduce

$PesterPreference = [PesterConfiguration]::Default
$PesterPreference.Output.Verbosity = 'Detailed'

$container = New-PesterContainer -ScriptBlock {
    BeforeAll {
        throw 'bad error'
    }
    Describe 'd1' {
        It 'i1' {
            1 | Should -Be 1
        }
        It 'i2' {
            1 | Should -Be 1
        }
        Describe 'd2' {
            It 'i3' {
                1 | Should -Be 1
            }
            It 'i4' {
                1 | Should -Be 1
            }
        }
    }
    Describe 'd3' {
        It 'i5' {
            1 | Should -Be 1
        }
        It 'i6' {
            1 | Should -Be 1
        }
    }
}

Invoke-Pester -Container $container

Describe your environment

Pester version : 6.0.0-alpha5 D:\git\Pester\bin\Pester.psm1 PowerShell version : 5.1.26100.3624 OS version : Microsoft Windows NT 10.0.26100.0

Possible Solution?

I'm open to helping contribute a fix here, probably changes in Output.ps1.

[fflaten]

In the console output, that is expected because a parent block failed. No need to write 100 noisy failures when it was a dependency that broke.

The xml outputs and resultobject (PassThru) should show them as failed.

[davidmatson]

Thanks, @fflaten.

We're relying on console output to tell us about everything that failed, so we can stream it into another system with full fidelity.

If we need this data while the tests are running, would #2581 be the better fix?

[nohwnd]

Yes, if you plug your code (as described in the last comment) you can stream all information that you select to stream.

[davidmatson]

@nohwnd - to clarify, are you referring to the comment on #2581, about using plugins?

[nohwnd]

yes, to adding this hook: https://github.com/pester/Pester/pull/2020

[davidmatson]

Thanks for the clarification, @nohwnd.

We have folks using Pester in a child process and hooking up standard output as the contract with the rest of the system. Last I was aware, plugins were not yet officially supported.

If there were a way to support output of failed test names like this via stdout, I think that could be very helpful. What would you think about showing these conditionally when ShowStartMarkers is enabled?