Feature Request: Have Describe block run in new PowerShell process

[SteveL-MSFT]

Context

For binary module authors, since PowerShell doesn't support unloading assemblies, once a module is imported, you have to start a new PowerShell process to test a new build of the dll. Currently, workaround is a wrapper PowerShell script that runs the Pester tests in a new process after importing the newly built assembly.

Proposal

It would be nice if there was a way to specify that the Describe block is run in a new process (could simply use Start-Job). BeforeAll would be used to do the import since the dll location will vary.

Debugging

Debugging will get more complicated because the tests are running in a new process. However, for tooling like Pester Test Explorer, if Invoke-Pester accepted a -CustomPipeName parameter which would be passed to pwsh -custompipename (which means Start-Job couldn't be used...) and then amended the script to use Wait-Debugger, then the tooling could attach to the right process before the tests were run.

cc @TylerLeonhardt

[TylerLeonhardt]

There will likely be work to be done in the PowerShell extension to light this up...

In Azure Functions we do something like leveraging a tasks.json and launch.json... I can almost get this working using a combo of that:

task.json:

{
          "label": "TestIt",
          "type": "shell",
          "command": "pwsh -c 'Import-Module ./build.psm1; Start-PSPester -Path ./test/powershell/Modules/Microsoft.PowerShell.Utility/ConvertTo-Csv.Tests.ps1'",
          "group": "test",
          "problemMatcher": "$pester",
          "isBackground": true
        }

and launch.json:

{
            "type": "PowerShell",
            "request": "attach",
            "name": "Attach to tests",
            "customPipeName": "pester",
            "runspaceId": 1,
        }

and modified Start-PSPester in the PowerShell repo to pass in a customPipeName.

but it hangs after it hits a breakpoint for some reason... could be the extension's fault... but even still it's all pretty messy.

I think we can get to a seamless experience here though.

[nohwnd]

I am open to this, but could we make it on file level (container level), or would that be too limiting? This way only the whole file could run in the new process, and we would not have to deal with re-running parent setups in the new process and so on...

[SteveL-MSFT]

Whole file would be fine as well, but then you have to run it via Invoke-Pester right? Was thinking that with Describe you can still just run the script directly.

[TylerLeonhardt]

Yeah we'd want the code lens in vscode to also work in this new mode. As long as that plays nicely with doing it at the file-level , then we should be fine

[TylerLeonhardt]

The PowerShell extension's script to debug a test will be more complicated now that we're talking about a subprocess.

[nohwnd]

Code lens is invoking the file from outside via Invoke-Pester. it would not work if you just F5 the file, there Describe parameter would be better. It in fact runs Invoke-Pester internally as well if you just run Describe by F5, so we could cram all the needed parameters there. We just need to make sure that we have all the info needed for the debugger to attach available in context somehow.