Provide an option to ignore flaky failed test in the report if the rerun test passes

[vlokesh09]

** Problem** I have 2-3 flaky tests in my cypress cucumber automation project. I usually get them working by using retries option in cypress which re runs the test if the test fails the first time. However , this cucumber html report is considering the rerun as an additional test.

For e.g.., if I have a total of 1 scenario and if it runs twice because I configured it to rerun on failing the first time , then cucumber report is reporting as

total tests : 2 passed : 1 Failed :1

Describe the solution you'd like I would like the option to have in cucumber reporter to ignore flaky tests so that the total number of tests will be 1, passed will be 1 from the above example

[wswebcreation]

Thanks for the feature request @vlokesh09

This reporter has no knowledge about the runs, it has not connection to the framework, it only translates what it gets.

There is a but, and that is that if you can determine that the previous run failed and was retried then we can think about implementing a "retry" status which we don't count. I'm not a big fan of this because it would be a difference between what is "officially" supported.

Love to hear your thoughts

[iliadelo]

hello all i manage to solve by just reformatting the generated json

`import fs from "fs"; let cucumberLogs = fs.readFileSync('cucumber-json/log.json'); let array = JSON.parse(cucumberLogs.toString()) array.forEach(obj=> { obj.elements = obj.elements.filter((item, index, self) => { let duplicate = self.findIndex((t) => t.id === item.id) === index; let failed = item.steps.some(s => s.result.status === "failed"); return !(duplicate && failed); });

})

fs.writeFileSync('cucumber-json/log.json', JSON.stringify(array)); `

you just create a file cucumberFormaterForRetries.mjs and then on your post test you just run the node script

maybe this can also be added to the source code

[cgarzon11]

Hi! Another possible solution could be tagging the first failing execution of the test as flaky (or skipped as it is tagged when you use testng and selenium) to differentiate three different statuses:

• Failed --> It has failed on the second try.
• Passed --> It has passed the first or second try.
• Skipped --> It has failed on the first try.

[AlfredoMoretta]

good afternoon guys, any news about this feature?