Pester icon indicating copy to clipboard operation
Pester copied to clipboard

Container is reported as Passed/Successful when discovery fails

Open fflaten opened this issue 1 year ago • 3 comments

Checklist

What is the issue?

When Discovery fails in a container after a valid test:

  • The blocks and tests up until the error is executed. Is this expected?
  • The block containing the discovery error is considered Passed
  • The container is reported as Passed even though it's listed under FailedContainers

The run is still reported as Failed and the error code returned from Invoke-Pester will be non-zero due to the failed container (set to 1 in repro).

Expected Behavior

Is this expected behavior? Should Container and block be successful when only partly discovered?

Steps To Reproduce

$sb = {
    Describe "d" {
        It '1' {}

        throw 'omg'
    }
}

$container = New-PesterContainer -ScriptBlock $sb
$r = Invoke-Pester -Container $container -PassThru

Describe your environment

Pester version : 6.0.0-alpha4 /workspaces/Pester/bin/Pester.psm1
PowerShell version : 7.4.1 OS version : Unix 5.15.153.1

Possible Solution?

No response

fflaten avatar Jul 09 '24 16:07 fflaten

The blocks and tests up until the error is executed. Is this expected+

Yes, originally I was failing just after discovery, but then it was very hard to migrate tests gradually because all tests had to be discoverable, instead of going test by test, file by file.

The block containing the discovery error is considered Passed

I think this should fail the block.

The container is reported as Passed even though it's listed under FailedContainers

Same here the container should fail.

nohwnd avatar Jul 10 '24 18:07 nohwnd

Thanks, that makes sense.

So fix the result values, but keep the partial container execution as-is?

fflaten avatar Jul 10 '24 18:07 fflaten

I think it makes sense that way, what do you think?

nohwnd avatar Jul 10 '24 19:07 nohwnd