Block setup and teardown executed when tests are skipped

[fflaten]

Checklist

• [X] Issue has a meaningful title
• [X] I have searched the existing issues. See all issues
• [X] I have tested using the latest version of Pester. See Installation and update guide.

What is the issue?

BeforeAll and AfterAll are executed when all tests are skipped using -Skip.

This is a known limitation/backlog item as seen in this runtime test: https://github.com/pester/Pester/blob/7889a88acd0ecd632d725185a061eab78b4e4c9b/tst/Pester.Runtime.ts.ps1#L1228-L1275

Expected Behavior

Setup and teardown are not executed when they're not needed

Note: Tests skipped using Set-ItResult doesn't count as they're skipped during Run-phase (too late)

Steps To Reproduce

$sb = {
    Describe 'A' {
        BeforeAll {
            Write-Host 'Before ALL -A'
        }
        BeforeEach {
            Write-Host 'Before EACH -A'
        }
        AfterEach {
            Write-Host 'After EACH -A'
        }

        It 'Test 1' -Skip {
            Write-Host 'This is test 1'
        }

        Describe 'B' {
            BeforeAll {
                Write-Host 'Before ALL -B'
            }
            AfterAll {
                Write-Host 'After ALL -B'
            }
            BeforeEach {
                Write-Host 'Before EACH -B'
            }
            AfterEach {
                Write-Host 'After EACH -B'
            }

            It 'Test 2' -Skip {
                Write-Host 'This is test 2'
            }
        }

        AfterAll {
            Write-Host 'After ALL -A'
        }
    }

}

$conf = New-PesterConfiguration
$conf.Output.Verbosity = 'Detailed'
$conf.Run.PassThru = $true
$conf.Run.ScriptBlock = $sb

$r = Invoke-Pester -Configuration $conf

Output:

Pester v5.6.0-beta1

Starting discovery in 1 files.
Discovery found 2 tests in 35ms.
Running tests.
Before ALL -A
Describing A
  [!] Test 1 6ms (0ms|6ms)
Before ALL -B
 Describing B
   [!] Test 2 3ms (0ms|3ms)
After ALL -B
After ALL -A
Tests completed in 77ms
Tests Passed: 0, Failed: 0, Skipped: 2, Inconclusive: 0, NotRun: 0

Describe your environment

Pester version : 5.6.0-beta1 /workspaces/Pester/bin/Pester.psm1
PowerShell version : 7.4.1 OS version : Unix 5.15.133.1

Possible Solution?

No response

[nohwnd]

I thought this was implemented, but maybe I've implemented it just on a smaller scope. Should we put this into 6.0.0 milestone?

[fflaten]

I think so. Just in case it affects something.

Not sure where to implement this yet. Mark block as Skip during PostProcess-DiscoveredBlock?

[fflaten]

Related #2424. If the plugin skips remaining tests the Before-/AfterAll should not run on subsequent blocks. That means we might need to make the decision during Run or both.

[nohwnd]

I don't remember where this is done, there was some function that is recursing down to figure out all the skipped steps, and then recursing up to mark all items as skipped, when they have all children skipped.