[Bug]: Playwright HAR file misses HTTP request headers when request is handled by service worker

[iphoneintosh]

trafficstars

Version

1.41.2

Steps to reproduce

When executing the following code, the first request installs a service worker, which then handles the second request. In this case, the HAR file entry of the second request that is handled by the service worker misses some (not all) HTTP request headers, such as Sec-Fetch-Dest.

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    context = p.chromium.launch_persistent_context(user_data_dir=None, record_har_path="/tmp/http.har", headless=False)
    page = context.new_page()
    page.goto("https://www.ukbusinessforums.co.uk") # installs service worker
    page.goto("https://www.ukbusinessforums.co.uk/register/connected-accounts/facebook/?setup=1")
    page.wait_for_load_state()
    context.close()

When executing the following code, the service worker is not installed and does not handle the request, and the HAR file entry contains all HTTP request headers as expected.

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    context = p.chromium.launch_persistent_context(user_data_dir=None, record_har_path="/tmp/http.har", headless=False)
    page = context.new_page()
    page.goto("https://www.ukbusinessforums.co.uk/register/connected-accounts/facebook/?setup=1")
    page.wait_for_load_state()
    context.close()

Expected behavior

It is expected that all requests in the HAR file, independent of whether they were handled by a service worker, contain all HTTP request headers that were actually sent to the server.

Actual behavior

All requests in the HAR file that were handled by a service worker are missing some HTTP request headers that were still sent to the server.

Additional context

No response

Environment

System:
    OS: macOS 14.3.1
    CPU: (14) arm64 Apple M3 Max
    Memory: 1.32 GB / 36.00 GB
  Binaries:
    Node: 21.6.1 - /opt/homebrew/bin/node
    npm: 10.2.4 - /opt/homebrew/bin/npm
  IDEs:
    VSCode: 1.86.1 - /opt/homebrew/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash

[mxschmitt]

Looks like this test covers it. Passes in Firefox, fails in WebKit and Chromium.

it('should be able to intercept a service worker intercepted document request', async ({ contextFactory, server }) => {
  server.setRoute('/something.html', (req, res) => {
    res.setHeader('Content-Type', 'text/html');
    res.end('From backend');
  });
  server.setRoute('/sw.js', (req, res) => {
    res.setHeader('Content-Type', 'application/javascript');
    res.end(`
      self.addEventListener('fetch', event => {
        event.respondWith(new Response('From service worker'));
      });
      self.addEventListener('activate', event => {
        event.waitUntil(clients.claim());
      });
    `);
  });
  const { page, getLog } = await pageWithHar(contextFactory, testInfo);
  await page.context().addCookies([{ name: 'foo', value: 'bar', url: server.PREFIX + '/something.html' }]);
  await page.goto(server.PREFIX + '/something.html');
  await page.evaluate(async () => {
    await navigator.serviceWorker.register('/sw.js');
    await new Promise(resolve => navigator.serviceWorker.oncontrollerchange = resolve);
  });
  await expect(page.getByText('From backend')).toBeVisible();
  await page.reload();
  await expect(page.getByText('From service worker')).toBeVisible();
  const log = await getLog();
  console.log(JSON.stringify(log, null, 2));
  const [first, second] = log.entries.filter(e => e.request.url.endsWith('something.html'));
  {
    // Normal request
    expect(first.request.url).toBe(server.PREFIX + '/something.html');
    expect(first.response.content.text).toBe('From backend');
    expect(first.request.cookies).toEqual([{ name: 'foo', value: 'bar' }]);
    expect(first.request.headers).toContainEqual({ name: 'Cookie', value: 'foo=bar' })
  }
  {
    // Service worker intercepted request
    expect(second.request.url).toBe(server.PREFIX + '/something.html');
    expect(second.response.content.text).toBe('From service worker');
    expect(second.request.cookies).toEqual([{ name: 'foo', value: 'bar' }]);
    expect(second.request.headers).toContainEqual({ name: 'Cookie', value: 'foo=bar' })
  }
});

[yury-s]

It is expected that all requests in the HAR file, independent of whether they were handled by a service worker, contain all HTTP request headers that were actually sent to the server.

There are two requests with different sets of headers. Headers added by the network service will only be present on the request that actually hits the network (if any, as service worker may handle the request without sending anything at all). What you see in the HAR file is a request sent by the page and handled by service worker. Such requests don't contain some headers that are added by the network service (e.g. cookies), the request doesn't have to be identical with the one which would be sent to the server, this is how the browsers work. HAR file does not contain requests sent by the service workers and it's a known Playwright issue.

In the example that @mxschmitt pasted above, no requests are sent to the server from the service worker, so there are actually no request with the cookies within the browser. That is working as intended.

I'll keep this bug open as we might reconsider support for recording requests sent by the service workers.

[pavelfeldman]

Why was this issue closed?

Thank you for your contribution to our project. This issue has been closed due to its limited upvotes and recent activity, and insufficient feedback for us to effectively act upon. Our priority is to focus on bugs that reflect higher user engagement and have actionable feedback, to ensure our bug database stays manageable.

Should you feel this closure was in error, please create a new issue and reference this one. We're open to revisiting it given increased support or additional clarity. Your understanding and cooperation are greatly appreciated.