vscode-powershell icon indicating copy to clipboard operation
vscode-powershell copied to clipboard

[Test] Switch to VSCode Extension Runner

Open JustinGrote opened this issue 1 year ago • 5 comments

Prerequisites

  • [X] I have written a descriptive issue title.
  • [X] I have searched all issues to ensure it has not already been reported.

Summary

VSCode now has a specific CLI and test runner extension for vscode E2E tests driven by Mocha https://code.visualstudio.com/api/working-with-extensions/testing-extension https://marketplace.visualstudio.com/items?itemName=ms-vscode.extension-test-runner

This will move our highly customized testing process to a more standard operation.

JustinGrote avatar Jan 04 '24 00:01 JustinGrote

That'd be awesome!

andyleejordan avatar Jan 04 '24 00:01 andyleejordan

@andyleejordan I'm evaluating it on the Pester Tests extension first and things are pretty promising, we should be able to port most of the process without much adjustment.

JustinGrote avatar Jan 04 '24 07:01 JustinGrote

After initial evaluations, this is feasible for the vscode-powershell extension but there are two areas our custom runner handles that vscode-test runner does not currently:

  1. Handling sourcemaps correctly when live-transpiling the tests
  2. Installing Prerequisite Extensions

I'll work with the vscode cli team a bit more, but their work more or less duplicates what we've already done with the custom runner so as long as all of our behavior can be incorporated there, the Mocha test runner extension they have devised is much nicer than our standard Mocha test runner hack, and would also allow us to more easily separate test suites into E2E (that require vscode runtime) and non-E2E that can be run more quickly.

JustinGrote avatar Jan 10 '24 20:01 JustinGrote

Amazing intel, thanks Justin!

andyleejordan avatar Jan 10 '24 20:01 andyleejordan

I've been able to move over my Pester Tests extension to use the cli and extension runner.

https://github.com/pester/vscode-adapter/pull/235

It's also able to run "pure" tests that don't require vscode code separately from the vscode ones.

Still has the limitations of not being able to pre-install extensions (workaroundable by pre-populating a profile) and does not work for transpling vscode e2e tests directly (sourcemaps dont work right) so the tests must be built, however I have a new esbuild config that can watch-build the tests without using TSC and work correctly with the test runner extension and still support source maps.

JustinGrote avatar Jan 17 '24 00:01 JustinGrote