[REGRESSION]: Tests fails when running in parallel in Firefox due to detached elements during dropdown click

[ilia-krivonosov]

Context:

• GOOD Playwright Version: 1.30
• BAD Playwright Version: 1.31 and above, up to 1.41
• Operating System: Linux or Mac

Code Snippet We are using the Mocha runner and launching the server in the mochaGlobalSetup global hook. Our setup is quite complex to reproduce, so I tried to create a similar one using the Playwright runner.
I have attached a link to the repository where there are pull requests with different versions of Playwright. There, you can see that the problem only occurs in Firefox and in Playwright versions from 1.31 to 1.41.

Links: Repository: https://github.com/ilia-krivonosov/pw-bug/pulls Console execution log: https://github.com/ilia-krivonosov/pw-bug/actions/runs/7585269348/job/20660738341?pr=4 Playwright report: https://github.com/ilia-krivonosov/pw-bug/actions/runs/7585269348?pr=4

Describe the bug After updating Playwright from version 1.30 to 1.40, we encountered an issue where a significant number of tests started failing when run on Firefox. The failures occur when attempting to click on an element in a dropdown, which results in the element getting detached. This issue only happens in Firefox when tests are run in parallel. We also noticed a correlation where the more threads we use, the more failures we encounter. It seems as if this happens due to a change in focus from the element.

[ilia-krivonosov]

I think they're related issues. https://github.com/microsoft/playwright/issues/27717

[abay-ibrayev]

Having the same issue for custom mocha runner. Need help please 🙏

[yury-s]

workers: process.env.CI ? 15 : 1, - you are running 15 browsers in parallel on a github actions bot which has only 2 cores. The tests start timing out as the machine doesn't keep up with the load, this is working as intended.

This issue only happens in Firefox when tests are run in parallel. We also noticed a correlation where the more threads we use, the more failures we encounter.

Yes, you cannot increase the number of parallel tests without consequences as there are resource limits. Firefox may just hit the limit earlier than others. Try reducing the number of workers to better match your hardware (Playwright usually picks right parallelism by default and you don't need to pass workers number manually).

[ilia-krivonosov]

you are running 15 browsers in parallel on a github actions bot which has only 2 cores. The tests start timing out as the machine doesn't keep up with the load, this is working as intended.

@yury-s I understand your point, but I believe the issue is not related to this. In our project, we are using a Jenkins agent with 64 cores and 128GB of RAM. I only used Github-actions as an example to reproduce the issue. I would also like to clarify that we are using the Mocha runner in our project, but it seems that the problem also occurs with the Playwright runner if it is launched in the mode with the browserServer startup and specifying the wsEndpoint in the configuration.

Try reducing the number of workers to better match your hardware

I will try to create another PR where I will reduce the number of threads to reproduce the issue.

[yury-s]

I will try to create another PR where I will reduce the number of threads to reproduce the issue.

We'd appreciate that. There may well be a bug in Playwright, but in the current repro it seems as a normal degradation when there are not enough resources.

[ilia-krivonosov]

@yury-s I made two more PRs in which I ran tests with 4 workers and without specific count of workers. It looks like the problem persists

with 4 workers https://github.com/ilia-krivonosov/pw-bug/actions/runs/7653803486/job/20856440205?pr=9

without specific count of workers (2 workers) https://github.com/ilia-krivonosov/pw-bug/actions/runs/7653854870/job/20856607176?pr=10

[dgozman]

Small repro that shows one page clicking, and another page stalling. It looks like clicks in different contexts do not happen in parallel.

import { firefox } from '@playwright/test';

const browser = await firefox.launch({ headless: false });

async function run(id) {
  const page = await browser.newPage();
  await page.setContent(`<button>Click me</button>`);
  for (let i = 0; i < 1000; i++) {
    console.log(`${id} click start #${i}`);
    await page.locator('button').click();
    console.log(`${id} click end #${i}`);
  }
}

run('page1');
run('page2');

[ilia-krivonosov]

@dgozman Do I understand correctly that the problem was fixed in version 1.43?

[dgozman]

@dgozman Do I understand correctly that the problem was fixed in version 1.43?

Yes, the fix has been released in v1.43.

[mikhail-shafigullin]

@dgozman @yury-s The error continues to recur. We have attempted to check it within this build on PW1.43 https://github.com/mikhail-shafigullin/pw-bug/actions/runs/8634232391/job/23669171822?pr=1 If necessary, I am ready to initiate a new issue. Repository: https://github.com/mikhail-shafigullin/pw-bug (it is fork of repository from topic starter)