webpack-dev-server
webpack-dev-server copied to clipboard
webpack
Puppeteer tests are flaky on CI
FAIL test/e2e/ClientOptions.test.js
● Client console.log › liveReload disabled (ws)
● Client console.log › liveReload & hot are disabled (ws)
● sockjs client port, no path › browser client › uses correct port and path
FAIL test/e2e/Socket-injection.test.js
● ws websocket client injection › testing when hot is enabled and liveReload is disabled › should be injected
Failed to launch the browser process!
Inconsistency detected by ld.so: ../elf/dl-tls.c: 481: _dl_allocate_tls_init: Assertion `listp->slotinfo[cnt].gen <= GL(dl_tls_generation)' failed!
TROUBLESHOOTING: https://github.com/puppeteer/puppeteer/blob/main/docs/troubleshooting.md
at onClose (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:193:20)
at ChildProcess.<anonymous> (node_modules/puppeteer/lib/cjs/puppeteer/node/BrowserRunner.js:184:79)
- https://github.com/webpack/webpack-dev-server/runs/1409566428
- https://github.com/webpack/webpack-dev-server/runs/1406463962
- https://github.com/webpack/webpack-dev-server/runs/1406343832
- https://github.com/webpack/webpack-dev-server/runs/1396967631
- https://github.com/webpack/webpack-dev-server/runs/1108036194
FAIL test/e2e/TransportMode.test.js
● transportMode client › custom client › on browser client › logs correctly
expect(received).toMatchSnapshot()
Snapshot name: `transportMode client custom client on browser client logs correctly 1`
- Snapshot - 5
+ Received + 1
Array [
"Hey.",
- "open",
- "liveReload",
- "[webpack-dev-server] Live Reloading enabled.",
- "hash",
- "ok",
+ "Failed to load resource: net::ERR_CONNECTION_REFUSED",
"close",
"[webpack-dev-server] Disconnected!",
]
82 | }
83 | }
> 84 | expect(res).toMatchSnapshot();
| ^
85 | done();
86 | });
87 | });
at test/e2e/TransportMode.test.js:84:33
FAIL test/e2e/TransportMode.test.js (16.054 s)
● transportMode client › sockjs › on browser client › logs correctly
expect(received).toMatchSnapshot()
Snapshot name: `transportMode client sockjs on browser client logs correctly 1`
- Snapshot - 1
+ Received + 1
Array [
"Hey.",
- "[webpack-dev-server] Live Reloading enabled.",
+ "Failed to load resource: net::ERR_CONNECTION_REFUSED",
"[webpack-dev-server] Disconnected!",
]
82 | }
83 | }
> 84 | expect(res).toMatchSnapshot();
| ^
85 | done();
86 | });
87 | });
at test/e2e/TransportMode.test.js:84:33
- https://github.com/webpack/webpack-dev-server/runs/1406731042
- https://github.com/webpack/webpack-dev-server/runs/1406126854
FAIL test/e2e/TransportMode.test.js (16.816 s)
● transportMode client › custom client › on browser client › logs correctly
● transportMode client › ws › on browser client › logs correctly
Protocol error (Page.navigate): Target closed.
58 | runBrowser().then(({ page, browser }) => {
59 | const res = [];
> 60 | page.goto(`http://localhost:${port}/main`);
| ^
61 | page.on('console', ({ _text }) => {
62 | res.push(_text);
63 | });
at node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:208:63
at CDPSession.send (node_modules/puppeteer/lib/cjs/puppeteer/common/Connection.js:207:16)
at navigate (node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:106:47)
at FrameManager.navigateFrame (node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:89:13)
at Frame.goto (node_modules/puppeteer/lib/cjs/puppeteer/common/FrameManager.js:405:41)
at Page.goto (node_modules/puppeteer/lib/cjs/puppeteer/common/Page.js:826:53)
at test/e2e/TransportMode.test.js:60:13
at runMicrotasks (<anonymous>)
- https://github.com/webpack/webpack-dev-server/runs/1406370533
- https://github.com/webpack/webpack-dev-server/runs/1406002795
ERROR: Failed to set up Chromium r782078! Set "PUPPETEER_SKIP_DOWNLOAD" env variable to skip download.
Error: getaddrinfo ENOTFOUND storage.googleapis.com
at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:66:26) {
errno: 'ENOTFOUND',
code: 'ENOTFOUND',
syscall: 'getaddrinfo',
hostname: 'storage.googleapis.com'
}
- https://github.com/webpack/webpack-dev-server/runs/1432768906
Maybe we can use docker to run test, better than local puppeteer to run
I've used Selenium in other projects and had a not bad experience. Its error rate is about one in two hundred.
Can we migrate to https://github.com/microsoft/playwright?
I haven't tried playwright, but the low number of issues seems nice.
playwright provides chromium, firefox, and webkit and we have some browser issues such as https://github.com/webpack/webpack-dev-server/issues/2466#issuecomment-706117588. yes so it's better to try it.
If somebody want to try it and help to migrate feel free to send a PR, I think it will be a good experience
@alexander-akait How should we do this, migrate all test at once or one by one in separate PRs then remove puppeteer in last?
I think we can do it in one PR, because API is almost same, we can encapsulate some helpers functions if it will make migration easy
@snitin315 @alexander-akait hey, I'm an applicant for the gsoc-2024. I started a PR for this issue which has been label relevant to gsoc as well. could you let me know this can be considered as a patch or an early contribution which is required in the proposal?
PR: https://github.com/webpack/webpack-dev-server/pull/5080/files
@alexander-akait hey, will this issue still be a project for gsoc-2024? I could not find it listed in the document attached on the gsoc website. I had opened a PR a few weeks ago here.
@mahdikhashan yes you could use this for a google project. We dont require you to use the ideas from the ideas list. This project is as good as any from the ideas document.
@alexander-akait would it be possible to share the invite link to Discord?
i think we should ensure Puppeteer and Chromium setup is correct, webpack-dev-server is running, handle any connection issues, and check for premature browser closures during navigation in test scripts.
