microsoft
[Feature]: UI Mode - Auto run tests
🚀 Feature Request
To be able to run tests automatically (instead of an explicit start) using UI mode
Detailed requirements:
- If set, works with
--uiflag or--ui-host/--ui-portflags - Respects test configuration (i.e. tests run the same as they would with
npx playwright test)
Example
npx playwright test --ui --auto-run
CLI Documentation:
--auto-run: 'When running in UI mode, starts running tests when first started"
Motivation
I want to be able to run this command and see the progress my tests on a CI
Today, the ui mode will load, but it requires going into the interface and clicking the ▶️ icon This would remove that extra step and runs the tests
I want to be able to run this command and see the progress my tests on a CI
What do you mean on a CI? There is no UI Mode on a CI, please use one of our reporters (HTML).
I want to be able to run this command and see the progress my tests on a CI
What do you mean on a CI? There is no UI Mode on a CI, please use one of our reporters (HTML).
Thank you for the reply, @pavelfeldman
I should rephrase. By CI, in my case, I mean a headless docker machine that I have control over (including networking).
On such a headless Docker environment, you can run --ui-host or --ui-port (reference)
The idea here is that with using the above option in a headless Docker environment, you will be able to see a live visual view into the running on the tests if you were to connect to this host/port from the the outside. Whereas with the HTML report as mentioned, you would only be able to see the results afterwards.
Please let me know if you have further questions and I'd be more than happy to elaborate
This is extremely valuable for long-running tests that you want to start on a remote server, e.g. triggered by CI, but have the possibility to login to the UI (through an externally exposed UI mode host and port) and check on the progress of these tests in the UI, and optionally restart failing ones.
Another use case is (a bit of an edge case): Some tests go through a "happy path" in 99% of the cases, but occasionally stop to ask for user interaction (fill a captcha or provide an MFA code). In such scenario a user needs to go and see the test suite and interact with a running test. The UI mode is helpful for such interactions. However, it should still launch the test in the first place when the CLI command is executed, so it doesn't have to wait for manual action in 100% of the cases.
It's awful UX to have a dev click a start button, when he was opening UI mode to commence the tests in the first place... when would someone want to open it, and not start the running of tests in it?
I'll add my +1 to this. Just setting up Playwright on a new project and couldn't figure out why my tests weren't going anywhere. Shocking that you have to find that tiny little button and then click it first. Take Vitest as a benchmark and just run it! Even Copilot thought the flag -ui-autorun was already available.
Just chiming in that I also think this would be useful. An optional flag to start UI mode AND run all the tests once open would be helpful when automating the test running process locally on dev machines. It'd be easier to debug all failed tests by having the UI already open.
