[BUG] Firefox startup issues on macOS

[bpasero]

Context:

• Playwright Version: 1.24.2
• Operating System: macOS
• Node.js version: 16.x
• Browser: Firefox
• Extra: running as part of our Azure Pipeline

We see our unit test fail when run on macOS with Firefox. this only started recently when we enabled running unit tests against Firefox:

2022-08-04T14:37:28.739Z pw:browser [pid=7361] <gracefully close start>
2022-08-04T14:37:28.815Z pw:browser [pid=7361] <process did exit: exitCode=0, signal=null>
2022-08-04T14:37:28.815Z pw:browser [pid=7361] starting temporary directories cleanup
2022-08-04T14:37:29.336Z pw:browser [pid=7361] finished temporary directories cleanup
2022-08-04T14:37:29.336Z pw:browser [pid=7361] <gracefully close end>
2022-08-04T14:37:29.393Z pw:browser <launching> /Users/runner/Library/Caches/ms-playwright/firefox-1335/firefox/Nightly.app/Contents/MacOS/firefox -no-remote -headless -profile /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/playwright_firefoxdev_profile-BnLp7D -juggler-pipe -silent
2022-08-04T14:37:29.424Z pw:browser <launched> pid=11728
2022-08-04T14:37:31.961Z pw:browser [pid=11728][err] *** You are running in headless mode.
2022-08-04T14:37:32.136Z pw:browser [pid=11728] <process did exit: exitCode=null, signal=SIGSEGV>
2022-08-04T14:37:32.136Z pw:browser [pid=11728] starting temporary directories cleanup
2022-08-04T14:37:32.140Z pw:browser [pid=11728] <gracefully close start>
2022-08-04T14:37:32.145Z pw:browser [pid=11728] finished temporary directories cleanup
2022-08-04T14:37:32.145Z pw:browser [pid=11728] <gracefully close end>
browserType.launch: Browser.enable): Browser closed.
==================== Browser output: ====================
<launching> /Users/runner/Library/Caches/ms-playwright/firefox-1335/firefox/Nightly.app/Contents/MacOS/firefox -no-remote -headless -profile /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/playwright_firefoxdev_profile-BnLp7D -juggler-pipe -silent
<launched> pid=11728
[pid=11728][err] *** You are running in headless mode.
[pid=11728] <process did exit: exitCode=null, signal=SIGSEGV>
[pid=11728] starting temporary directories cleanup
=========================== logs ===========================
<launching> /Users/runner/Library/Caches/ms-playwright/firefox-1335/firefox/Nightly.app/Contents/MacOS/firefox -no-remote -headless -profile /var/folders/24/8k48jl6d249_n_qfxwsl6xvm0000gn/T/playwright_firefoxdev_profile-BnLp7D -juggler-pipe -silent
<launched> pid=11728
[pid=11728][err] *** You are running in headless mode.
[pid=11728] <process did exit: exitCode=null, signal=SIGSEGV>
[pid=11728] starting temporary directories cleanup
============================================================
    at runTestsInBrowser (/Users/runner/work/1/s/test/unit/browser/index.js:129:48)
    at /Users/runner/work/1/s/test/unit/browser/index.js:254:25 {
  name: 'Error'
}
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
##[error]Bash exited with code '1'.

Build: https://dev.azure.com/monacotools/Monaco/_build/results?buildId=180612&view=logs&j=55ac390b-ffb1-50ea-3650-525dd9a3fd80&t=3a692518-c3cf-5f6a-c273-72f71dc185fb&l=16933

[mxschmitt]

I had a discussion with the team about it. So basically Firefox fails to launch in your scenario, we are able to reproduce it, not often, but maybe 1/50 or 1/100 of launches.

We'll dig into it.

What could be done as a workaround is to add retries to the tests itself, which we do inside our Playwright Test (@playwright/test) test-runner by default, so it does not surface that often. Since your tests are inside the page, you'd need to wrap the "launch(), newContext(), newPage() logic" and retry it up to 3 times while we dig into fixing the actual issue.

[bpasero]

Thanks! We try to avoid retrying as much as possible because its typically a bandaid and not a fix.

[pavelfeldman]

Mac looks good to me at this point, but we need to check Firefox Windows.

[drk-mtr]

Just to confirm this also fails on Windows, using the template project created by doing npm init playwright@latest, without modifications. For me this isn't inconsistent i.e. it fails every time. Chrome, Chromium, Edge and Safari tests run fine using the same setup.

• @playwright/test - version 1.26.1
• Firefox - version 105.0.1
• Windows 10 - version 10.0.19044

If there is any way I can post logs that will provide additional useful info just let me know.

[pavelfeldman]

Why was this issue closed?

Thank you for your contribution to our project. This issue has been closed due to its limited upvotes and recent activity, and insufficient feedback for us to effectively act upon. Our priority is to focus on bugs that reflect higher user engagement and have actionable feedback, to ensure our bug database stays manageable.

Should you feel this closure was in error, please create a new issue and reference this one. We're open to revisiting it given increased support or additional clarity. Your understanding and cooperation are greatly appreciated.