allure-js
allure-js copied to clipboard
Results of Scenario Outline are presented as a one test instead of separate results for every row in the Examples section
Describe the bug Results of Scenario Outline are presented as a one test instead of separate results for every row in the Examples section. It seems that all executions of Scenario Outline are interpreted as Retries of one Scenario.
To Reproduce Steps to reproduce the behavior:
- Create a project with devDependencies:
"@cucumber/cucumber": "^9.0.0", "allure-cucumberjs": "^2.0.0" - Prepare feature file with a Scenario Outline and an Examples section
- Execute the command
npm test - Execute the command
allure server allure-results
Sample code: https://github.com/DanielDelimata/sample-typescript-cucumberjs-selenium Sample results: https://danieldelimata.github.io/sample-typescript-cucumberjs-selenium
Correctly working code for older versions: https://github.com/DanielDelimata/sample-typescript-cucumberjs-selenium-old Correct results: https://danieldelimata.github.io/sample-typescript-cucumberjs-selenium-old/
Expected behavior The number of tests cases should reflect the number of rows in Examples section. Every row defines separate test case so each result should be presented separately.
Screenshots
Observed results:

Expected results:

Additional context
From my observation, the Allure framework works correctly for Cucumber in Java, C#, Python, Kotlin. The problem may be related to changes in Cucumber itself (precisely in js package @cucumber/cucumber) cause for old cucumber versions (e.g. 6.0.5) it works correctly (see e.g. https://github.com/allure-examples/allure-javascript-cucumberjs/blob/master/package.json). It does not work correctly for versions above 7.
This is happening on all integrations of this version Also on a integration of playWright https://github.com/Tallyb/cucumber-playwright
Any chance this can be picked up? I upgraded today from cucumber 7 to 9 and starting having this issue. To be fair, I was using "allure-cucumberjs8": "^2.0.0-beta.15", which had its own issues but at least test were displaying correctly.
Update: I just tried to run Cucumber9 with the beta version of allure-cucumberjs8 and this is not a bug there so definitely not a cucumber issue.
The same problem, it's a bug. And as a workaround, I had to add a variable to the scenario header.
Fixed in 3.0.0-beta.11