cucumber-reports-plugin icon indicating copy to clipboard operation
cucumber-reports-plugin copied to clipboard

mergeFeaturesWithRetest option in Jenkins cucumber-report plugin does not work

Open tarasmytlovych opened this issue 1 year ago • 1 comments

Jenkins and plugins versions report

Environment
cucumber-reports:5.8.0

What Operating System are you using (both controller, and any agents involved in the problem)?

Linux Ubuntu, playwright

Reproduction steps

Plugin configs:

cucumber buildStatus: "UNSTABLE", fileIncludePattern: '**/*.json', jsonReportDirectory: 'ui-tests/target/test-results', skipEmptyJSONFiles: true, pendingAsNotFailingStatus: true, mergeFeaturesById: true, mergeFeaturesWithRetest: true, sortingMethod: 'ALPHABETICAL', classifications: [ [ 'key' : 'Environment', 'value': 'QA' ] ]

tests run generates 2 json files: cucumber.json and cucumber-rerun.json

Expected Results

When test is passed during rerun, in report it should be shown once, the latest attempt and the pipeline should not fail because of the first attempt failure

Actual Results

The scenario is shown in the report twice, failed and passed tries Pipeline is marked as failed even though all failed tests have passed after the rerun

Anything else?

No response

Are you interested in contributing a fix?

No response

tarasmytlovych avatar Jan 03 '24 10:01 tarasmytlovych

I have found a workaround - just remove mergeFeaturesById and leave only mergeFeaturesWithRetest.

This removed the duplicated scenarios from the report, but the Jenkins pipeline is still marked as failed even though all tests pass after rerun

But would like to have some fix to keep both these options

tarasmytlovych avatar Jan 03 '24 12:01 tarasmytlovych