Add ability to rerun failed/skipped tests

[chuckfossen]

A command line parameter similar to --start-at="suite/test" that would run all the subsequent tests starting at that particular test. That would allow the continuation of the rest of the tests instead of starting over.

[Snaipe]

I think that if such an option was to be included, it shouldn't depend on the ordering. Criterion doesn't guarantee any ordering for tests between versions, and parallelized tests aren't really helping that either. It's true that for convenience w.r.t the implementation of --list the runner is currently sorting those tests, but this is an implementation detail more than anything.

Instead, a more useful behaviour would be to re-run tests that previously failed, or tests that did not have any time to run (when --fail-fast is specified), in my opinion.

[chuckfossen]

I wouldn't care about ordering between versions. Just that they would be deterministic between subsequent runs when I'm iterating on fixing issues in my tests.

[Snaipe]

Depending on the ordering would also be prevending us from implementing test randomization or run-order optimization in the future, so I'd say implementing a --rerun-failed would be the way to go rather than a --start-at=foo/bar.