Skip remaining repeated tests on failure

[dirk-thomas]

The rerunfailures plugin can be used to detect flaky tests and ignore them by trying to rerun previous failed tests. If a repeated run succeeds that is then considered "good enough".

I am interested in the "opposite" feature and thought it might fit into this plugin. To ensure that a test is not flaky I want to run it multiple times. This plugin already supports that use case. In the case where a test fails I don't want the remaining invocations to happen (in order to save time). Using the pytest option -x is not feasible for that case since I am still interested in the result of all different tests and don't want to abort testing all together.

Would an additional option like --stop-repeating-same-test-if-it-failed-once (just a name to describe the semantic) fit into this plugin? If yes, with a little pointer I might be able to provide a pull request if that is helpful.

[AdamWRichardson]

Is there any update on this? Would also be very interested in such a feature!

[Martiix]

I am also interested in this, except I want to set a limit to how many times an individual test can fail before it stops running this test and continues with the rest. Pytest has the --maxfail attribute, but I would like to combine it with python-repeat, therefore needing it on an individual test. I'm looking for something like @pytest.mark.repeat(100) @pytest.mark.maxfail(10) def test(): Any updates?