playwright icon indicating copy to clipboard operation
playwright copied to clipboard

[Bug]: Playwright frequently fails to download Chromium

Open gundermanc opened this issue 6 months ago • 7 comments

Version

I believe latest -- via playwright-mcp 0.0.29

Steps to reproduce

  1. Enable the Playwright MCP server in a host, such as GitHub Copilot coding agent.
  • Coding agent docs: https://docs.github.com/en/copilot/using-github-copilot/coding-agent
  • Enabling playwright: https://docs.github.com/en/copilot/using-github-copilot/coding-agent/extending-copilot-coding-agent-with-mcp#example-playwright
  1. Prompt the host to make a minimal react app and then take a screenshot and add it to the PR description. The copilot coding agent runs in a minimal environment with little pre-installed. When it reaches the step to use the Playwright MCP server, it first has to bootstrap by installing chromium.

I'm posting this here instead of in the playwright-mcp repo because the actual download failure seems to be in playwright-core package.

Expected behavior

I expect the playwright-core library to consistently download and install Chromium.. and/or retry on failure. Perhaps retry behavior should be configurable for unattended CI scenarios such as Copilot Coding agent.

Actual behavior

This step often fails with the following stack:

Error: Download failed: size mismatch, file size: 179968518, expected size: 0 URL: https://playwright.download.prss.microsoft.com/dbazure/download/playwright/builds/chromium/1178/chromium-linux.zip
    at WriteStream.<anonymous> (/home/runner/work/gundermanc-playground/gundermanc-playground/node_modules/playwright-core/lib/server/registry/oopDownloadBrowserMain.js:69:24)
    at WriteStream.emit (node:events:536:35)
    at finish (node:internal/streams/writable:955:10)
    at node:internal/streams/writable:936:13
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
Failed to install browsers
Error: Failed to download Chromium 138.0.7204.15 (playwright build v1178), caused by
Error: Download failure, code=1
    at ChildProcess.<anonymous> (/home/runner/work/gundermanc-playground/gundermanc-playground/node_modules/playwright-core/lib/server/registry/browserFetcher.js:94:32)
    at ChildProcess.emit (node:events:524:28)
    at ChildProcess._handle.onexit (node:internal/child_process:293:12)

The repro is unfortunately not 100% consistent but it did seemingly reproduce more consistently with the playwright-mcp 0.0.29.

Additional context

No response

Environment

This runs in a sandboxed environment, but the details are approximately similar to that of WSL 24.04.1 LTS

  System:
    OS: Linux 6.6 Ubuntu 24.04.2 LTS 24.04.2 LTS (Noble Numbat)
    CPU: (20) x64 Intel(R) Core(TM) i9-10900 CPU @ 2.80GHz
    Memory: 24.86 GB / 31.20 GB
    Container: Yes
  Binaries:
    Node: 22.15.0 - /usr/local/bin/node
    Yarn: 1.22.22 - /mnt/c/Users/chgund/AppData/Roaming/npm/yarn
    npm: 10.9.2 - /usr/local/bin/npm
    pnpm: 10.12.1 - /mnt/c/Users/chgund/AppData/Roaming/npm/pnpm
  IDEs:
    VSCode: 1.101.1 - /mnt/c/Users/chgund/AppData/Local/Programs/Microsoft VS Code/bin/code
  Languages:
    Bash: 5.2.21 - /usr/bin/bash

gundermanc avatar Jun 23 '25 17:06 gundermanc

That's interesting. expected size: 0 points towards the HTTP response missing a content-length header:

https://github.com/microsoft/playwright/blob/896cb8536e7ca561fd9a6179cbb9643cccc848bc/packages/playwright-core/src/server/registry/oopDownloadBrowserMain.ts#L73-L84

This could be a change in the CDN configuration. I'll investigate.

Skn0tt avatar Jun 24 '25 09:06 Skn0tt

I have a hunch this is related to Copilot Agent traffic running through an HTTP proxy, that strips out the Content-Length header. I'll look at improving our error handling on our end. Could you check the behaviour of your proxy as well? It's a common bug in proxies that they change a stream to Transfer-Encoding: chunked, but then erroneously drop the Content-Length header.

Skn0tt avatar Jun 24 '25 09:06 Skn0tt

I have a hunch this is related to Copilot Agent traffic running through an HTTP proxy, that strips out the Content-Length header

I asked around and no one is aware of a proxy being used in this scenario. There is a firewall used with the runtime but it's not currently enabled for the Playwright MCP server.

gundermanc avatar Jun 24 '25 16:06 gundermanc

I can see Playwright CDN urls mentioned in the Copilot "Allow list":

Image

https://github.com/microsoft/playwright/actions/runs/15850725159/job/44683254181

Are you sure this isn't related? This is the first report we're getting for this kind of download failure.

Skn0tt avatar Jun 24 '25 16:06 Skn0tt

So far, what we've confirmed in the one repro we have (which is slightly different from the reported bug), is that the go proxy solution we have is setting the content-length header correctly. There is progress bar logic in the chrome download code of playwright core that seems to be dividing by 0 https://github.com/microsoft/playwright/blob/1c3488aa5f0454135ca954fd8828bc0935e1a468/packages/playwright-core/src/server/registry/browserFetcher.ts#L163

In the error logs it is pointing to the console.log statement; however in the code line 163 is suggesting it a divide by 0 caused Infinity to be passed to a string repeat in JS which causes a RangeError.

 RangeError: Invalid count value: Infinity
              at String.repeat (<anonymous>)
              at /root/.npm/_npx/e41f203b7505f1fb/node_modules/playwright-core/lib/server/registry/browserFetcher.js:163:32

Right above this error we added the following log statement locally to show that the proxy is setting a non-zero value for the content-length

msg="HTTP proxy handling response" content_length=179923316 content_length_header=179923316

logs put together

          time=2025-06-24T17:18:45.017Z level=INFO msg="HTTP proxy handling response" content_length=179923316 content_length_header=179923316
          /root/.npm/_npx/e41f203b7505f1fb/node_modules/playwright-core/lib/server/registry/browserFetcher.js:163
                console.log(`|${"\u25A0".repeat(row * stepWidth)}${" ".repeat((totalRows - row) * stepWidth)}| ${percentageString}% of ${toMegabytes(totalBytes)}`);
                                         ^
          
          RangeError: Invalid count value: Infinity
              at String.repeat (<anonymous>)
              at /root/.npm/_npx/e41f203b7505f1fb/node_modules/playwright-core/lib/server/registry/browserFetcher.js:163:32
              at ChildProcess.<anonymous> (/root/.npm/_npx/e41f203b7505f1fb/node_modules/playwright-core/lib/server/registry/browserFetcher.js:90:7)
              at ChildProcess.emit (node:events:518:28)
              at emit (node:internal/child_process:949:14)
              at process.processTicksAndRejections (node:internal/process/task_queues:91:21)
          
          Node.js v22.16.0
❌ Expected exit code 0 but got 1

The content length looks correct to me

@mgrosen ➜ /workspaces/testing $ curl https://playwright.download.prss.microsoft.com/dbazure/download/playwright/builds/chromium/1179/chromium-linux.zip --output chromium-linux.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  171M  100  171M    0     0   211M      0 --:--:-- --:--:-- --:--:--  211M
@mgrosen ➜ /workspaces/testing $ ls -l chromium-linux.zip 
-rw-rw-rw- 1 root root 179923316 Jun 24 17:59 chromium-linux.zip

FWIW, I'm still seeing that its working to download without our firewall, so I think there is a very subtle bug here causing the calculations to be off.

@Skn0tt do you know how I can get the logs that go to the DebugLogger? I've tried specifying the DEBUG_FILE env var several ways and it hasn't started writing to the file. I think that would be helpful to figure out exactly what values are seen by the browserFetcher logic

mgrosen avatar Jun 24 '25 18:06 mgrosen

I found out how to ... get some of the logs, and it appears there is some point between the response leaving the proxy and when the downloadPackages receives the response that something happening, because logs from the two areas disagree.

 time=2025-06-24T19:20:13.441Z level=INFO msg="HTTP proxy handling response" content_length=179923316 content_length_header=179923316 
      response_headers = {
          Accept-Ranges: [bytes]
          Age: [94642]
          Cache-Control: [public, max-age=259200]
          Connection: [keep-alive]
          Content-Disposition: [attachment; filename=chromium-linux.zip; filename*=UTF-8''chromium-linux.zip]
          Content-Length: [179923316]
          Content-Type: [application/octet-stream]
          Date: [Tue, 24 Jun 2025 19:20:13 GMT]
          Etag: ["0x8AF8D11F025BE09D34B962ADF55D0F6BE63C8A2506F533A2139E24635DB49389"]
          Last-Modified: [Thu, 12 Jun 2025 13:33:16 GMT]
          Via: [1.1 varnish]
          X-Cache: [HIT]
          X-Cache-Hits: [3]
          X-Ccc: [US]
          X-Cid: [3]
          X-Ms-Apiversion: [Distribute 1.2]
          X-Ms-Region: [prod-eus-z1]
          X-Osid: [2]
          X-Served-By: [cache-bfi-krnt7300044-BFI]
          X-Timer: [S1750792813.375306,VS0,VE0]
      }
          install -- response status code: 200
          install -- content-headers: {
            "accept-ranges": "bytes",
            "age": "94642",
            "cache-control": "public, max-age=259200",
            "connection": "close",
            "content-disposition": "attachment; filename=chromium-linux.zip; filename*=UTF-8''chromium-linux.zip",
            "content-type": "application/octet-stream",
            "date": "Tue, 24 Jun 2025 19:20:13 GMT",
            "etag": "\"0x8AF8D11F025BE09D34B962ADF55D0F6BE63C8A2506F533A2139E24635DB49389\"",
            "last-modified": "Thu, 12 Jun 2025 13:33:16 GMT",
            "transfer-encoding": "chunked",
            "via": "1.1 varnish",
            "x-cache": "HIT",
            "x-cache-hits": "3",
            "x-ccc": "US",
            "x-cid": "3",
            "x-ms-apiversion": "Distribute 1.2",
            "x-ms-region": "prod-eus-z1",
            "x-osid": "2",
            "x-served-by": "cache-bfi-krnt7300044-BFI",
            "x-timer": "S1750792813.375306,VS0,VE0"
          }
          install -- total bytes: 0

teasing out the repsonse hea

It is possible that the value is being set to zero by the proxy we are using AFTER the on response handle runs, I'll dive into that to see if there's any chance thats happening

mgrosen avatar Jun 24 '25 19:06 mgrosen

I found the logic where the content-length header is being dropped. TL;DR there is logic in the proxy implementation we use that is deleting the content length header (and setting the connection header to close), and thats happening after any of our custom logic. So I don't see any indication of an issue in playwright code, thanks for getting involved!

mgrosen avatar Jun 24 '25 23:06 mgrosen

Turns out my hunch was right! There it is in the headers, Content-Length dropped but Transfer-Encoding: chunked added :D I worked on CDNs in my previous job, so this kind of subtle proxy bug i've seen time and again. I'll go ahead and close since there's no action from Playwright.

Skn0tt avatar Jun 25 '25 07:06 Skn0tt

@Skn0tt I was looking into this more, and I realize the reason that the proxy changes to chunked is so that it doesn't load the entire stream into memory before passing it on to the client. Is there any option to turn off the progress bar logic? Or would it be possible to update the logic in browserFetcher.js, etc. to respect chunked encoding?

mgrosen avatar Jun 25 '25 16:06 mgrosen

Our download logic is in our critical path and has worked fine for years, so we'd prefer not to touch it unless necessary. Fixing this in Playwright is the wrong place, you'll run into similar issues with other systems trying to download things.

Not chunking the response body doesn't prevent you from streaming it. Transfer-Encoding: chunked is for when you don't know the body size in advance, which often comes together with application-level streaming (think live video feeds or server-sent-events), but you don't need to buffer the proxied message without it - you can just stream it through.

Skn0tt avatar Jun 26 '25 07:06 Skn0tt

So what's the solution/workaround for the playwright installation issue in a copilot environment? We are seeing a similar error when copilot tries to run browser tests

[playwright install] Downloading Chromium 140.0.7339.16 (playwright build v1187) from https://cdn.playwright.dev/dbazure/download/playwright/builds/chromium/1187/chromium-linux.zip
[playwright install] /home/runner/work/azure-sdk-for-js/azure-sdk-for-js/node_modules/.pnpm/[email protected]/node_modules/playwright-core/lib/server/registry/browserFetcher.js:163
[playwright install]       console.log(`|${"\u25A0".repeat(row * stepWidth)}${" ".repeat((totalRows - row) * stepWidth)}| ${percentageString}% of ${toMegabytes(totalBytes)}`);
[playwright install]                                ^
[playwright install] 
[playwright install] RangeError: Invalid count value: Infinity
[playwright install]     at String.repeat (<anonymous>)
[playwright install]     at /home/runner/work/azure-sdk-for-js/azure-sdk-for-js/node_modules/.pnpm/[email protected]/node_modules/playwright-core/lib/server/registry/browserFetcher.js:163:32
[playwright install]     at ChildProcess.<anonymous> (/home/runner/work/azure-sdk-for-js/azure-sdk-for-js/node_modules/.pnpm/[email protected]/node_modules/playwright-core/lib/server/registry/browserFetcher.js:90:7)
[playwright install]     at ChildProcess.emit (node:events:524:28)
[playwright install]     at emit (node:internal/child_process:950:14)
[playwright install]     at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
[playwright install] 
[playwright install] Node.js v20.19.5
[playwright install] node:events:502
[playwright install]       throw er; // Unhandled 'error' event
[playwright install]       ^
[playwright install] 
[playwright install] Error: write EPIPE
[playwright install]     at target._send (node:internal/child_process:878:20)
[playwright install]     at target.send (node:internal/child_process:751:19)
[playwright install]     at progress (/home/runner/work/azure-sdk-for-js/azure-sdk-for-js/node_modules/.pnpm/[email protected]/node_modules/playwright-core/lib/server/registry/oopDownloadBrowserMain.js:36:17)
[playwright install]     at IncomingMessage.onData (/home/runner/work/azure-sdk-for-js/azure-sdk-for-js/node_modules/.pnpm/[email protected]/node_modules/playwright-core/lib/server/registry/oopDownloadBrowserMain.js:92:5)
[playwright install]     at IncomingMessage.emit (node:events:536:35)
[playwright install]     at Readable.read (node:internal/streams/readable:782:10)
[playwright install]     at flow (node:internal/streams/readable:1283:53)
[playwright install]     at resume_ (node:internal/streams/readable:1262:3)
[playwright install]     at process.processTicksAndRejections (node:internal/process/task_queues:82:21)
[playwright install] Emitted 'error' event on process instance at:
[playwright install]     at node:internal/child_process:882:39
[playwright install]     at process.processTicksAndRejections (node:internal/process/task_queues:77:11) {
[playwright install]   errno: -32,
[playwright install]   code: 'EPIPE',
[playwright install]   syscall: 'write'
[playwright install] }
[playwright install] 
[playwright install] Node.js v20.19.5
[playwright install] npx playwright install exited with code 1

jeremymeng avatar Sep 18 '25 00:09 jeremymeng

As a temporary work around you can try this. In copilot setup steps, you can pre-install the browsers required with something like npx playwright install _browsers_. Its not a great solution, but hopefully this will unblock agent experience immediately for you.

mgrosen avatar Sep 29 '25 17:09 mgrosen

As a temporary work around you can try this. In copilot setup steps, you can pre-install the browsers required with something like npx playwright install _browsers_. Its not a great solution, but hopefully this will unblock agent experience immediately for you.

@mgrosen thanks for the suggestion! I forgot to mention that the above error is from us doing a npx playwright install before the browser tests are running https://github.com/Azure/azure-sdk-for-js/blob/92c512e99056101ee3ff1d6377e06c1056c2e013/common/tools/dev-tool/src/commands/run/testVitest.ts#L49-L59.

Do you think the environment of copilot setup steps would be different than when copilot is executing its tasks? I will also give it a try

jeremymeng avatar Sep 29 '25 19:09 jeremymeng

If the downloader receives a response without Content-Length, it will fail either at the size check or, if there's no TTY attached as in Jeremy's case, it will fail while showing progress. Based on that, i'd be surprised if the workaround helped.

@mgrosen this is related to CCA's Firewall, right? How is that imposed onto Copilot? I verified it's not via the HTTP_PROXY env var. Are you using a locally trusted cert to intercept outgoing traffic?

Skn0tt avatar Sep 30 '25 06:09 Skn0tt

Rant into this myself today, would love to fix it. @mgrosen @jeremymeng can you help me understand what's going on here, so we can figure out the correct fix?

Skn0tt avatar Oct 08 '25 09:10 Skn0tt

Also seeing this error, which is probably related:

Image

Skn0tt avatar Oct 08 '25 09:10 Skn0tt

Fixed through https://github.com/microsoft/playwright/pull/37832.

Skn0tt avatar Oct 17 '25 08:10 Skn0tt

This is common in corporate environments with ssl inspection, the common approach is to tell node to accept the self-signed certificate

NODE_TLS_REJECT_UNAUTHORIZED=0 npx playwright install

tribet84 avatar Nov 06 '25 12:11 tribet84