New Plugin Extension to Find Flaky Tests (Enhancement PR)

[yashsaboo]

Order-Dependent(OD) tests are flaky tests whose results can differ depending on the order in which the tests run. An order-dependent test consistently passes when run in one order but then consistently fails when run in a different order. [Definition and plugin extension inspiration from Research Paper]

We have come up with a new extensions to this plugin that helps in identifying the OD Flaky test which features two flags: --flaky-test-finder=<repetition_number> where repetition_number > 1 --flaky-test-log-path=<log_path>

• --flaky-test-finder When this flag is enabled, it identifies the flaky test that satisfies the above definition.
• --flaky-test-log-path This flag specifies where the flaky test logs are flushed.

Contributors: Yash Saboo Nirupam K N

[jbasko]

Thank you for the submission. I will have to spend some time on this to consider the merit of including this in the project. I might not be able to find the time before Christmas. Feel free to remind here if there is no update by then.

[Nirupamkn]

That sounds great! Merry Christmas :) @jbasko

[jbasko]

Apologies for taking 3 years to look again into this. I think your contribution does not belong to this simple plugin.