playwright
playwright copied to clipboard
microsoft
[Bug]: Completed test fails on tearing down "context"
Version
1.41.1
Steps to reproduce
Successful test fails in teardown
in afterAll only ws client disconnect presents ( which completes successfully )
Similar https://github.com/microsoft/playwright/issues/24210 - but there were failing test so trace: 'retain-on-failure', shouldn't bother
const config: PlaywrightTestConfig = {
globalSetup: require.resolve("./global-setup"),
use: {
// headless: false,
ignoreHTTPSErrors: true,
screenshot: "off",
video: "off",
trace: "retain-on-failure",
},
projects: [
{
name: "Desktop-Chrome",
use: {
browserName: "chromium",
channel: "chrome",
viewport: { width: 1920, height: 1080 },
launchOptions: {
args: ["--disable-web-security"],
// slowMo: 1000,
},
},
},
],
expect: {
timeout: 15000,
},
reporter: [["list"], ["html", { open: "never" }], ["./customReporter.ts"]],
reportSlowTests: { max: 0, threshold: 240000 },
timeout: 230000,
globalTimeout: 2500000,
forbidOnly: !!env.CI,
}
Expected behavior
tests not failing on standard fixtures teardown
Actual behavior
Test finished within timeout of 230000ms, but tearing down "context" ran out of time.
Please allow more time for the test, since teardown is attributed towards the test timeout budget.
from the test steps seen, there was plenty of time for teardown :( happens rarely, but brings frustration
Additional context
No response
Environment
gitlab CI
+
FROM mcr.microsoft.com/playwright:v1.41.1-jammy
+
ARG CHROME_VERSION="121.0.6167.85-1"
@dimkin-eu I think this is still related to tracing. Even though there were no errors, the trace was still recorded (just in case there would be a failure), and tracing is stopped during the teardown. Let me think what we can do about it. From the timing on your screenshot, it looks like teardown takes the vast majority of the execution time, so increasing the timeout is unlikely to help.
We are encountering the same issue. Sometimes the teardown of a test takes an unexpectedly long time
Same here. It would be great if I don't have to use 'on-first-retry' - retries sweep bugs under the rug. Sometimes there are bugs caused by race conditions - e.g. one request happens to arrive before the other and then it reproduces. I want to catch them and that trace is a mighty tool!
Playwright released the v1.42, so I updated mine and it solved this issue.
Playwright released the v1.42, so I updated mine and it solved this issue.
I hope so, will try
To everyone: if you encounter this issue with the recent 1.42 version, please share a repro so we can debug it.
@dgozman I've not been able to perfectly replicate this issue, but started seeing this error when I added a context.on('request') to my test.extend.
I've created a minimal example here - https://github.com/joeyciechanowicz/playwright-context-teardown
Though it doesn't timeout, when you comment out the context.on in api.ts the After Hooks run considerably faster. ~2s without a .on and ~7-10s with.
Those same lines of code were causing the teardown to timeout on my test suite, presumably because it was much closer to the 30s time limit already
@joeyciechanowicz Unfortunately, I cannot repro this on my computer - with/without the listener each test executes in ~1s.
Fascinating. Any docs/guides on how I could dig into this a bit?
For reference I'm on an M1 Mac.
I am also using M1 Mac if it helps. I didn't see this issue for a while, will add steps from the failed test when I see it.
Fascinating. Any docs/guides on how I could dig into this a bit?
Not really. You can try profiling the tests by running PWTEST_PROFILE_DIR=./profiles npx playwright test and then load json files in Performance panel in Chrome DevTools. Perhaps you'll see something interesting, but that's not very likely.
For reference I'm on an M1 Mac.
Same here, but the behavior is different.
I am closing this issue, because we were not able to get a repro. If anyone can share a reliable public repro, please file a new issue by filling in the "Bug Report" template.