Add retry feature for failed tests

[theredfish]

WIP

[theredfish]

@ilslv I think what I want to implement is very similar to WhichScenarioFn in runner::Basic. I started to explore an implementation with this signature :

pub type RetryableFn = fn(&gherkin::Feature, &gherkin::Scenario) -> Option<u8>;

The implementation will :

1. Check if there is a tag retry(n) on the feature, parse the digit and save it in a mutable variable retries. If there is no tag, we set the variable to None.
2. Check if there is a tag on the scenario and mutate the variable to Some(n) if any. Since the scenario prevails on the feature. If not do nothing.
3. Last check if n is 0 then mutate the variable to None (if anything different from unsigned, should be None due to bad parsing from the regex)

Regarding the user input (the retry tag) I don't think we have a clean way to prevent bad inputs. So everything that doesn't match the regex will be None and not retried.

Also I will try to extract this closure to its own function so I can do some unit tests. But I didn't explore this yet. I suppose I need to keep this function RetryableFn public so it can be extended by custom runners?

For the next steps :

• I will explore a solution to keep the retries associated to scenarios in memory
• For each retry, launch the scenario again (question : maybe we should add a default timeout between retries with tokio::time::sleep for example? But in case of async it could probably be the source of bugs + I'll need to adapt it for @serial scenarios).

[ilslv]

@theredfish

I think what I want to implement is very similar to WhichScenarioFn in runner::Basic

Yes, this functionality should definitely be in Runner. And WhichScenarioFn is a good starting point. I imaging RetryableFn looking something like this:

enum RetryOrder {
     /// Reties `Scenario` right after the failure.
     Immediately,
     /// Retries `Scenario` in the end. 
     Postponed,
}

pub type RetryableFn = fn(
    &gherkin::Feature,
    Option<&gherkin::Rule>,
    &gherkin::Scenario,
) -> (RetryOrder, usize);

Also we should modify the existing Scenarios storage to also contain number of retries:

https://github.com/cucumber-rs/cucumber/blob/cf055ac06c7b72f572882ce15d6a60da92ad60a0/src/runner/basic.rs#L1404-L1429

type RetiesLeft = usize;
type IsRetried = bool;

type Scenarios = HashMap<
    ScenarioType,
    Vec<(
        Arc<gherkin::Feature>,
        Option<Arc<gherkin::Rule>>,
        Arc<gherkin::Scenario>,
        RetriesLeft,
        RetryOrder,
        IsRetried, // Used to emit right events
    )>,
>;

So we should re-insert failed Scenarios in the front or back based on the RetryOrder value.

For each retry, launch the scenario again (question : maybe we should add a default timeout between retries with tokio::time::sleep for example? But in case of async it could probably be the source of bugs + I'll need to adapt it for @serial scenarios).

This should be discussed separately, as currently we are runtime-agnostic and introducing tokio dependency would break it. Maybe this would be possible with separate feature enabled, but this is definitely out of the current PRs scope.

[theredfish]

Thank you for your inputs @ilslv ! I will follow them for the implementation and see how to implement the events 👍

[theredfish]

Closing in favor of #223