microsoft
[Bug]: Tests fail for code using media query `display-mode: fullscreen` under new Chromium headless
Version
1.50.1
Steps to reproduce
Reproduction at https://github.com/unikitty37/chromium-headless-issue
- Opt in to new Chromium headless, as detailed in #33566
- Create a Svelte5 + SvelteKit site that uses a reactive media query to detect if the browser is fullscreen (
const fullscreen = new MediaQuery('display-mode: fullscreen')) - Add an element to a page that only displays if the browser is not fullscreen (
{#if !fullscreen.current}<element />{/if}) - Create a test that goes to the page, calls
page.evaluate(() => document.documentElement.requestFullscreen())and then checks to see if the element from step 3 is visible
Expected behavior
Both tests should pass whether running under old Chromium headless or new Chromium headless.
Actual behavior
Both tests pass using the old Chromium headless, but the second test fails when running the new Chromium headless.
Running 4 tests using 2 workers
✓ 1 …5:5 › Full screen control › when the browser is not full screen › is displayed (322ms)
✓ 2 …5:5 › Full screen control › when the browser is not full screen › is displayed (356ms)
✓ 3 …8:5 › Full screen control › when the browser is full screen › is not displayed (299ms)
✘ 4 …18:5 › Full screen control › when the browser is full screen › is not displayed (5.3s)
1) [new-chromium] › e2e/demo.test.js:18:5 › Full screen control › when the browser is full screen › is not displayed
Error: Timed out 5000ms waiting for expect(locator).not.toBeVisible()
Locator: getByRole('button', { name: 'Full Screen' })
Expected: not visible
Received: visible
Call log:
- expect.not.toBeVisible with timeout 5000ms
- waiting for getByRole('button', { name: 'Full Screen' })
9 × locator resolved to <button>Full Screen</button>
- unexpected value "visible"
17 |
18 | test('is not displayed', async ({ page }) => {
> 19 | await expect(page.getByRole('button', { name: 'Full Screen' })).not.toBeVisible()
| ^
20 | })
21 | })
22 | })
at /Users/user/Development/chromium-headless-issue/e2e/demo.test.js:19:75
1 failed
[new-chromium] › e2e/demo.test.js:18:5 › Full screen control › when the browser is full screen › is not displayed
3 passed (6.5s)
Additional context
No response
Environment
System:
OS: macOS 14.7.1
CPU: (10) arm64 Apple M1 Max
Memory: 241.28 MB / 64.00 GB
Binaries:
Node: 23.6.0 - /opt/homebrew/bin/node
Yarn: 1.22.22 - /opt/homebrew/bin/yarn
npm: 10.9.2 - /opt/homebrew/bin/npm
pnpm: 9.12.1 - ~/bin/pnpm
IDEs:
VSCode: 1.76.1 - /Users/user/bin/code
Languages:
Bash: 3.2.57 - /bin/bash
npmPackages:
@playwright/test: ^1.49.1 => 1.50.1
The behaviour also occurs when running in a devcontainer:
System:
OS: Linux 6.12 Debian GNU/Linux 12 (bookworm) 12 (bookworm)
CPU: (10) arm64 unknown
Memory: 8.84 GB / 11.73 GB
Container: Yes
Binaries:
Node: 22.13.0 - /usr/local/share/nvm/versions/node/v22.13.0/bin/node
Yarn: 1.22.22 - /usr/bin/yarn
npm: 10.9.2 - /usr/local/share/nvm/versions/node/v22.13.0/bin/npm
pnpm: 9.15.4 - /usr/local/share/nvm/versions/node/v22.13.0/bin/pnpm
IDEs:
VSCode: 1.97.0 - /vscode/vscode-server/bin/linux-arm64/33fc5a94a3f99ebe7087e8fe79fbe1d37a251016/bin/remote-cli/code
Languages:
Bash: 5.2.15 - /usr/bin/bash
npmPackages:
@playwright/test: 1.50.1 => 1.50.1
I tried to run your repro and get:
WebServer] src/routes/+page.svelte (3:11): "MediaQuery" is not exported by "node_modules/.pnpm/[email protected]/node_modules/svelte/src/reactivity/index-server.js", imported by "src/routes/+page.svelte".
do you mind looking into it?
@mxschmitt Thanks for taking a look — it seems to have picked a more recent version of Svelte when I built it. I've changed the package.json to use explicit version numbers, so hopefully we'll both be running the same versions of everything now…
I can reproduce. Test:
// tests/page/page-emulate-media.spec.ts
import { expect as baseExpect, test, Page } from '@playwright/test'
const expect = baseExpect.extend({
async toMatchMedia(page: Page, mediaQuery: string) {
const pass = await page.evaluate(mediaQuery => matchMedia(mediaQuery).matches, mediaQuery).catch(() => false);
return {
message() {
if (pass)
return `Expected "${mediaQuery}" not to match, but it did`;
else
return `Expected "${mediaQuery}" to match, but it did not`;
},
pass,
name: 'toMatchMedia',
};
},
});
test('Full screen control is displayed', async ({ page }) => {
await page.goto('https://example.com')
await expect(page).not.toMatchMedia('(display-mode: fullscreen)')
})
test('Full screen control is not displayed', async ({ page }) => {
await page.goto('https://example.com')
await page.evaluate(() => document.documentElement.requestFullscreen())
await expect(page).toMatchMedia('(display-mode: fullscreen)')
})
Passes in Chromium Headless Shell but not with 'channel': 'chromium'.
Minimal Puppeteer reproduction:
import puppeteer from 'puppeteer';
(async () => {
for (const headless of [true, 'shell']) {
for (const callSetFocusEmulationEnabled of [true, false]) {
const browser = await puppeteer.launch({ headless });
const page = await browser.newPage();
const props = [
headless === true ? 'Headless New' : 'Headless Shell',
callSetFocusEmulationEnabled ? 'Emulation.setFocusEmulationEnabled' : 'no Emulation.setFocusEmulationEnabled',
`Browser: ${await browser.version()}`,
`Platform: ${process.platform}`,
]
console.log()
console.log(props.join(' | '));
// Problematic code: Emulation.setFocusEmulationEnabled is not getting used in Puppeteer
// by default hence the issue does not surface there. Playwright uses this always.
const session = await page.createCDPSession();
if (callSetFocusEmulationEnabled)
await session.send(`Emulation.setFocusEmulationEnabled`, { enabled: true });
const verifyFullscreenMode = async (mode, expected) => {
const pass = await page.evaluate(
(mediaQuery) => matchMedia(mediaQuery).matches,
'(display-mode: fullscreen)'
);
const passString = pass === expected ? 'passed' : 'failed';
console.log(`- ${mode}: (${passString}) - Expected: ${expected}, Actual: ${pass}`);
};
await page.goto('https://example.com');
await verifyFullscreenMode('Without fullscreen mode', false);
await page.evaluate(() => document.documentElement.requestFullscreen());
await verifyFullscreenMode('With fullscreen mode', true);
await browser.close();
}
}
})();
Expected: Pass on all 4 variations.
Actual: Headless New with Emulation.setFocusEmulationEnabled behaves differently and '(display-mode: fullscreen)' does not return true after calling document.documentElement.requestFullscreen().
Headless New | Emulation.setFocusEmulationEnabled | Browser: Chrome/135.0.7049.42 | Platform: darwin
- Without fullscreen mode: (passed) - Expected: false, Actual: false
- With fullscreen mode: (failed) - Expected: true, Actual: false
Headless New | no Emulation.setFocusEmulationEnabled | Browser: Chrome/135.0.7049.42 | Platform: darwin
- Without fullscreen mode: (passed) - Expected: false, Actual: false
- With fullscreen mode: (passed) - Expected: true, Actual: true
Headless Shell | Emulation.setFocusEmulationEnabled | Browser: HeadlessChrome/135.0.7049.42 | Platform: darwin
- Without fullscreen mode: (passed) - Expected: false, Actual: false
- With fullscreen mode: (passed) - Expected: true, Actual: true
Headless Shell | no Emulation.setFocusEmulationEnabled | Browser: HeadlessChrome/135.0.7049.42 | Platform: darwin
- Without fullscreen mode: (passed) - Expected: false, Actual: false
- With fullscreen mode: (passed) - Expected: true, Actual: true
Bad headless shell version:
- "version": "133.0.6862.0"
- "revision": "1388591"
- https://crrev.com/1388591
Good headless shell version:
- "version": "133.0.6863.0"
- "revision": "1388808"
- https://crrev.com/1388808
RANGE: https://chromium.googlesource.com/chromium/src/+log/30886f7231b93c289fffcfc85dc2f2dca59f4430..bfbe6e71d59705ae9366e2efbc9f44b8cad8d2e8
Suspect which makes the test pass: https://chromium.googlesource.com/chromium/src/+/e616fc75b5ce7aa8e9d4db3bf19d4ca3307146d1
Goal: We'd like to have something similar in Headless New.
Investigation notes: as per upstream Chromium issue, this is a side-effect of not actually making page fullscreen. This is intended behavior in most cases, because making headed (or new headless) browser fullscreen during testing prevents user for doing anything. It is unclear whether there is a proper solution. I'll merge this into the larger fullscreen issue.
