[Bug]: Webkit tests are 2-3x slower than Chrome or Firefox

[pascalknupper]

trafficstars

Version

1.45.0

Steps to reproduce

Tests in Webkit seem to be slower than usual recently. The same set of tests takes round about 2-3x in webkit than running in chrome.

Tests are running in "mcr.microsoft.com/playwright:v1.45.0-jammy" docker container. We are also heavily make use of the visual comparison.

I´ll see if I can prepare a repo for reproduction.

Expected behavior

Performance in Webkit is improved

Actual behavior

Tests in Webkit are 2-3x slower than in Chrome or Firefox

Additional context

No response

Environment

System:
    OS: Linux 5.15 Ubuntu 22.04.4 LTS 22.04.4 LTS (Jammy Jellyfish)
    CPU: (6) arm64 unknown
    Memory: 6.55 GB / 7.67 GB
    Container: Yes
  Binaries:
    Node: 20.15.0 - /usr/bin/node
    Yarn: 1.22.22 - /usr/bin/yarn
    npm: 10.7.0 - /usr/bin/npm
  Languages:
    Bash: 5.1.16 - /usr/bin/bash
  npmPackages:
    @playwright/test: ^1.45.0 => 1.45.0

[StanislavGrishaev]

take a look at this - https://github.com/microsoft/playwright/issues/18119#issuecomment-1370734489 it helped me

[pavelfeldman]

There isn't much we can do without the repro. Are you saying it is a regression relative to 1.44?

[pascalknupper]

thanks @StanislavGrishaev I´ll see whether this configuration change will improve our test performance in webkit

[mxschmitt]

I'll close it for now, if you have a reproduction repository, feel free to re-file and linking to this one. Thanks for your understanding!

(re above, we recommend tracing over video, since its much more powerful and requires less CPU/memory).

[gvillo]

For the records, I found a workaround to avoid failed tests (because of timeout) with the release of v1.52.0 that introduces testProject/workers

The idea is to set 1 worker for every webkit project and I found the sweet spot for now.

Example:

{
	name: 'iPhone 15 Safari Guest Flow',
	dependencies: ['setup'],
	grep: [/@guest/],
	use: {
		...devices['iPhone 15'],
	},
	workers: process.env.CI ? 1 : undefined,
}