Just Question; Is it possible to rerun by `--onlyFailures` when the playwright fail to connect to the browser?

[aha-oretama]

Describe the bug This is not a bug, just a question.

I use --onlyFailures to make our tests more stable. We can use it for no-watch mode now. https://github.com/facebook/jest/pull/10678

I use Moon which provides a browser grid.

Sometimes, our test failed to connect the browser for some reasons... For example, it takes much time for the browser to stand up and the connection timeout occurs.

In this situation, when executing the following command, jest-playwright doesn't rerun the failed test 😢

$ jest --onlyFailures

I'm glad that jest-playwright helps to rerun the failed tests when the connection error occurs.

Question: Are there any workarounds to rerun the failed tests by --onlyFailures when the playwright fails to connect to the browser? Or do you have any plan to support for this feature?

To Reproduce

Use this jest-playwright.config.js

module.exports = {
  launchType: 'SERVER',
  connectOptions: {
    wsEndpoint: "ws://wrong-url"
  }
};

Execute following commands

$ jest
$ jest --onlyFailures

Expected behavior jest-playwright helps to rerun the failed tests when the connection error occurs.

Desktop (please complete the following information):

• OS: macOS
• Playwright version 1.8.1
• jest-playwright version 1.4.7

[mmarkelov]

@aha-oretama not sure but maybe retryTimes can help

[aha-oretama]

I tried... but it doesn't work 😓

This is my sample code. I confirmed that the test can be retried when there is no connection error.

jest.retryTimes(3);

beforeEach(async () => {
  await jestPlaywright.resetBrowser();
  await page.goto('https://whatismybrowser.com/')
})

test('should display correct browser', async () => {
  const browser = await page.$eval('.string-major', (el) => el.innerHTML)
  expect(true).toBe(false);
  expect(browser).toContain('Chrome')
})

But, when the connection error occurs, the test is never retried... 😢 The log is like this.

  ● Test suite failed to run

    browserType.connect: getaddrinfo ENOTFOUND wrong-url. Most likely ws endpoint is incorrect

      at WebSocket.<anonymous> (node_modules/playwright/lib/client/browserType.js:158:28)