playwright
playwright copied to clipboard
microsoft
[Feature] Component testing: use vite dev server mode instead of production mode
When I run component tests, I found first uses rollup to bundle the files and then run the test, I wonder if we can use vite dev server mode to run the component tests, then we have many benefits from vite:
- watch mode
- faster start up,
- support HMR mode
- faster feedback
dev server actually makes things much, much slower. each test is making an endpoint request and with dev mode each request takes 200ms. so we deliberately made a decision to go prod mode. Once you have 100 tests, 200ms per request becomes a 20 seconds of delay non-starter.
This feature is excellent, especially for debugging. My project's build takes 30-40 seconds, and if I change something and want to debug my test, I need to wait for that time. Maybe it could be some env variable that turns on that behavior? 🤔
Another addition that I thought. It would be great to combine vite serve + some watch mode or test restart. I changed my code, and one test or some tests restarted immediately.
I agree with @AlexanderMykulych, in my case I found it took me about 20 sec to finish the prod build. If we don't use dev server mode, then how are we going to support watch mode, HMR and TDD, and getting faster feedback?
Same feature request. In my case debugging errors with React in the production build is painfully slow.
My current approach is to use vite server, and create an index.html and index.tsx file for my tests, which is a bit of manual work. However it gives me a very fast feedback loop as I can run a component test almost without delay by pointing playwright at the vite server. When using playwright component testing, I have to wait for the full prod build which as mentioned above is very slow, though perfect for CI as the tests run the fastest they can - this leaves the dev experience lacking.
Hi @cameronbraid , do you have an example how to set it up?
Its just using normal vite running in dev mode and using playwright to drive the browser to the vite pages
Just create a new folder for each test, with the following as an example
src/test-a/index.html
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
</head>
<body>
<div id="root" style="height:100%"></div>
<script type="module" src="./index.tsx"></script>
</body>
</html>
src/test-a/index.tsx
import * as React from "react"
import { createRoot } from "react-dom/client"
const TestComponent = () => {
return <div data-testid='TestComponent'>TestComponent</div>
}
createRoot(document.getElementById("root")).render(<TestComponent/>)
src/test-a/test-Playwright.tsx
import { expect, Request, Route, test } from "@playwright/test"
import { viteUrl } from "src/viteUrl"
test.describe("TestComponent", () => {
test("renders", async ({ page }) => {
await page.goto(viteUrl(`/src/test-a/`))
await expect(page.locator("[data-testid=TestComponent]").toContainText("TestComponent"))
})
})
src/viteUrl.ts
export function viteUrl(path: string) {
return (process.env.VITE_URL || "http://localhost:3002") + path
}
As I said above it is a bit of manual work
Cameron
Would love to have a faster option that doesn't require a full prod build. 🙏
Any plans to improve the iteration rate?
Right now, we have to wait for tens of seconds of vite production build time for each small change to the source code, for watch or --ui mode.
It makes TDD super painful for component testing under playwright. We have to stick to Vitest + RTL for component testing.
Vite has plans to improve production builds by creating a faster, Rust-based replacement for the Rollup bundler. This will likely improve iteration speed/performance significantly. See @yyx990803's recent keynote for more info: https://youtu.be/hrdwQHoAp0M?si=DwWDdMWh7_zz51fv&t=501
What would be the preferred shape of a fix for this?
(a) Watch mode
# blocking process
npm run test -- --watch
- it would build the app for test, start the dev server with it
- every time component or test is changed, all the affected tests are executed
(b) Dev server
# blocking process
npm run test -- --dev
- it would build the app for test, start the dev server with it
- every time user runs
npm run testtests run against the dev server
(c) Something else?
Out of curiosity - will this change make the following scenarios possible:
- Running playwright using a user-defined (test-focused) set of vite's environment variables (or even a custom mode)
- Running playwright tests on an SSR app where outgoing requests from the server-side part can be caught and mocked with tools like msw
I've been looking for ways to test a SvelteKit app E2E with playwright, but I need ways to catch and mock outgoing requests from the server-side part. Haven't found a solution so far and I've wondered whether playwright using vite's dev server might indeed give me a way to achieve this.
@spuxx1701 there is a separate issue to track this: https://github.com/microsoft/playwright/issues/19399.
I've been looking for ways to test a SvelteKit app E2E with playwright, but I need ways to catch and mock outgoing requests from the server-side part.
See: https://github.com/microsoft/playwright/issues/19399#issuecomment-1642171000
What would be the preferred shape of a fix for this?
https://github.com/microsoft/playwright/issues/21960 is what i was looking for and https://github.com/microsoft/playwright/issues/23372 if i have to use VSCode. Would that be the same as option A?
This approach enables me to keep the browser open alongside the editor, allowing for a more efficient workflow as changes to the component or test are automatically reflected in the browser without the necessity of executing npm run test repeatedly. I haven't had much time to look into this, but it seems like there are still some performance gains to be made. Especially for larger projects (see: https://github.com/microsoft/playwright/issues/14748#issuecomment-1152051894). Getting as close as possible to the snappy Vite dev server/partial build/HMR experience would be great as this also allows component preview/development in isolation.
Not sure if I understand the pro's and cons of option B correctly. It seems that i still have to execute npm run test manually every time i modify the component or test?
Not sure if I understand the pro's and cons of option B correctly. It seems that i still have to execute npm run test manually every time i modify the component or test?
When assessing the VS Code workflow, think about this feature as of two separate features that result in the "watch" behavior:
- Dev server that keeps component gallery up-to-date live. After (before, or during) your test run, editing component will result in component changing live in the "show browser" window. Dev server must outlive the test run, i.e. must outlive the playwright test session.
- Watch re-run the tests upon certain event, this is the watch feature in VS Code that we have not implemented yet. It is the same for e2e and for components module the dependency tree.
(1) is already implemented on ToT:
# start the server
npx pw-react dev-server
Then you just run the tests (console or run decoration in VS Code) and they will use the server.
Hi there, I was wondering if there are any updates or new developments regarding this issue? Thank you for your continued work on this 🙌