Provide a way to re-run/retry an entire spec upon failure

[abhidp]

What would you like?

Ability to rerun/retry an entire spec file upon failure This is different from retrying a failed it() block.

Currently, retries: n in cypress.json only retries the failed it() function. Sometime due to server issues, these retrying mechanism on individual it() blocks are not enough. Also this retry mechanism doesn't retry the before() hooks.

I want to re-run the entire spec file once again from beginning including all hooks. WebdriverIO has a way of running an entire specFile via their config file: https://webdriver.io/docs/retry/#add-retries-on-a-per-specfile-basis

Why is this needed?

Sometimes, the backend server might be re-booting or unresponsive. At that exact time, when tests run , they will fail as the server is not ready. Here the retry mechanism will not be of much help as it retries the same thing again and again, although there is no response coming from the backend. When we run the same spec again manually, it passes, because by that time, the server has recovered. This is a big false positive from the testing side.

In such cases, I want to retry the entire spec as many times as I want (via a config setting as mentioned above, similar to webdriverIO).

Re-running those failed specs will prevent failed builds due to flaky servers.