Pester clears test drives inefficiently

[splatteredbits]

Checklist

• [x] Issue has a meaningful title
• [x] I have searched the existing issues. See all issues
• [x] I have tested using the latest version of Pester. See Installation and update guide.

What is the issue?

After each Context block runs, Pester clears the test drive. Unfortunately, it does this really inefficiently by calling Remove-Item on each item in the test drive. One of my test fixtures downloads and unpackages multiple copies of Node.js, each which contains over 5,000 files. The way Clear-TestDrive removes files, it takes about 30 seconds to delete all the files (on Linux specifically). If it called Remove-Item instead, it would take less than a second.

Expected Behavior

I would expect the test drive to be cleared way more efficiently.

Steps To Reproduce

This is the closest I can get to a reproduction. In this example, it takes pester around 4 seconds to clear the test drive. In my actual tests, it takes Pester around 25 to 30 seconds.

BeforeAll {
    function InitNode
    {
        param(
            [String] $In
        )

        Write-Verbose "$(Get-Date)  ${In}" -Verbose
        $pkgPath = Join-Path -Path $In -ChildPath 'node-v22.18.0-linux-x64.tar.xz'
        Invoke-WebRequest -Uri 'https://nodejs.org/dist/v22.18.0/node-v22.18.0-linux-x64.tar.xz' `
                          -OutFile $pkgPath
        tar -xJf $pkgPath -C $In

        $pkgPath = Join-Path -Path $In -ChildPath 'node-v16.20.2-linux-x64.tar.xz'
        Invoke-WebRequest -Uri 'https://nodejs.org/dist/v16.20.2/node-v16.20.2-linux-x64.tar.xz' `
                          -OutFile $pkgPath
        tar -xJf $pkgPath -C $In

        $numv22 = 1
        for ($i = 0; $i -lt $numv22; ++$i)
        {
            $testDirPath = Join-Path -Path $In -ChildPath $i
            New-Item -Path $testDirPath -ItemType Directory
            $linkPath = Join-Path -Path $testDirPath -ChildPath '.node'
            $linkTarget = Join-Path -Path $In -ChildPath 'node-v22.18.0-linux-x64'
            New-Item -Path $linkPath -ItemType SymbolicLink -Value $linkTarget
            New-Item -Path (Join-Path -Path $testDirPath -ChildPath '.output') -ItemType Directory
            New-Item -Path (Join-Path -Path $testDirPath -ChildPath 'PSModules') -ItemType Directory
            New-Item -Path (Join-Path -Path $testDirPath -ChildPath 'package.json') -value ('p' * 3)
            New-Item -Path (Join-Path -Path $testDirPath -ChildPath 'whiskey.yml') -Value ('w' * 27)
        }

        $numv16 = 5
        for ($i = 0; $i -lt $numv16; ++$i)
        {
            $testDirPath = Join-Path -Path $In -ChildPath ($i + $numv22)
            New-Item -Path $testDirPath -ItemType Directory
            $linkPath = Join-Path -Path $testDirPath -ChildPath '.node'
            $linkTarget = Join-Path -Path $In -ChildPath 'node-v16.20.2-linux-x64'
            if ($i -lt 4)
            {
                New-Item -Path $linkPath -ItemType SymbolicLink -Value $linkTarget
            }
            New-Item -Path (Join-Path -Path $testDirPath -ChildPath '.output') -ItemType Directory
            New-Item -Path (Join-Path -Path $testDirPath -ChildPath 'PSModules') -ItemType Directory
            New-Item -Path (Join-Path -Path $testDirPath -ChildPath 'whiskey.yml') -Value ('w' * 27)
        }
    }
}

Describe 'Clear-TestDrive' {
    BeforeEach {
        Write-Verbose "$(Get-Date) BeforeEach" -Verbose
    }

    AfterEach {
        Write-Verbose "$(Get-Date) AfterEach" -Verbose
        Write-Verbose "$((Get-ChildItem -Path $TestDrive -Recurse | Measure-Object).Count) items in TestDrive." -Verbose
    }

    Context "using TestDrive" {
        It 'deletes files' {
            InitNode -In $TestDrive
        }
    }

    Context 'using custom temp path' {
        BeforeEach {
            $tempPath = Join-Path -Path ([IO.Path]::GetTempPath()) -ChildPath ([IO.Path]::GetRandomFileName())
            New-Item -Path $tempPath -ItemType Directory
        }

        AfterEach {
            Write-Verbose "$(Get-Date)  Deleting " -Verbose
            $ProgressPreference = 'SilentlyContinue'
            Remove-Item -Path $tempPath -Recurse -Force
            Write-Verbose "$(Get-Date)  Deleting Complete" -Verbose
        }

        It 'deletes files' {
            InitNode -In $tempPath
        }
    }
}

I modified Clear-TestDrive to perf test it:

function Clear-TestDrive ([String[]]$Exclude, [string]$TestDrivePath) {
    if ([IO.Directory]::Exists($TestDrivePath)) {

        $timer = [Diagnostics.Stopwatch]::StartNew()

        Remove-TestDriveSymbolicLinks -Path $TestDrivePath

        $count = 0
        $errCount = 0
        foreach ($i in [IO.Directory]::GetFileSystemEntries($TestDrivePath, "*.*", [System.IO.SearchOption]::AllDirectories)) {
            if ($Exclude -contains $i) {
                continue
            }

            & $SafeCommands['Remove-Item'] -Force -Recurse $i -ErrorAction SilentlyContinue -ErrorVariable 'removeItemErrors'
            $count += 1
            $errCount += ($removeItemErrors | Measure-Object).Count
        }
        Write-Verbose "$(Get-Date)  Clear-TestDrive  Deleted ${count} items from ${TestDrivePath} in $($timer.Elapsed) with ${errCount} errors." -Verbose
    }
}

Describe your environment

Pester version     : 5.4.1 /mnt/c/Build/PWSH-Whiskey/PSModules/Pester/5.4.1/Pester.psm1
PowerShell version : 7.5.1
OS version         : Unix 6.6.87.2

Possible Solution?

It would be nice if Pester just Get-ChildItem -Path $TestDrive | Remove-Item -Recurse -Force -ErrorAction Ignore

It would be nice if we could tell Pester how to delete somehow as parameters to blocks, something like:

Context 'some context' -TestDriveTearDownBehavior DoNotClear {
}

Describe 'Some Function' -TestDriveTearDownBehavior IgnoreErrors,UseRemoveItem,OneByOne {
}

[splatteredbits]

Here's a screenshot of my actual tests running using Pester's TestDrive with perf info:

[splatteredbits]

And here's a screenshot running using my own temp directory:

[nohwnd]

Interesting that this is not accounted into the overhead timer, I guess I just accounted the code, and not extensions.

And I see the problem. I am not sure how to solve it though and keep the current functionality of test drive where we clear new files that were added between tests.

[splatteredbits]

Interesting that this is not accounted into the overhead timer, I guess I just accounted the code, and not extensions.

Maybe have a level of output beyond Detailed that also outputs timings for each BeforeEach/AfterEach/Context blocks as well?

[splatteredbits]

And I see the problem. I am not sure how to solve it though and keep the current functionality of test drive where we clear new files that were added between tests.

Why not use Remove-Item to remove each item in the test drive?

function Clear-TestDrive ([String[]]$Exclude, [string]$TestDrivePath) {
    if ([IO.Directory]::Exists($TestDrivePath)) {
         & $SafeCommands['Get-ChildItem'] -Path $TestDrivePath | & $SafeCommands['Remove-Item'] -Force -Recurse -ErrorAction Ignore
    }
}

[splatteredbits]

If you notice from my perf output, Clear-TestDrive is spending the majority of its time deleting items that have already been deleted:

Deleted 16396 items from TESTDRIVE in 00:00:41.1227357 with 16390 errors.

So it is deleting 6 items recursively from the test drive directory itself, but then proceeding to delete all the other files that were deleted by the first 6 calls to Remove-Item.

[nohwnd]

Aah okay, I did not motice that.