allure-js icon indicating copy to clipboard operation
allure-js copied to clipboard

Results of Scenario Outline are presented as a one test instead of separate results for every row in the Examples section

Open DanielDelimata opened this issue 2 years ago • 3 comments

Describe the bug Results of Scenario Outline are presented as a one test instead of separate results for every row in the Examples section. It seems that all executions of Scenario Outline are interpreted as Retries of one Scenario.

To Reproduce Steps to reproduce the behavior:

  1. Create a project with devDependencies: "@cucumber/cucumber": "^9.0.0", "allure-cucumberjs": "^2.0.0"
  2. Prepare feature file with a Scenario Outline and an Examples section
  3. Execute the command npm test
  4. Execute the command allure server allure-results

Sample code: https://github.com/DanielDelimata/sample-typescript-cucumberjs-selenium Sample results: https://danieldelimata.github.io/sample-typescript-cucumberjs-selenium

Correctly working code for older versions: https://github.com/DanielDelimata/sample-typescript-cucumberjs-selenium-old Correct results: https://danieldelimata.github.io/sample-typescript-cucumberjs-selenium-old/

Expected behavior The number of tests cases should reflect the number of rows in Examples section. Every row defines separate test case so each result should be presented separately.

Screenshots Observed results: Pasted Graphic 1 Pasted Graphic 2

Expected results: Pasted Graphic

Additional context From my observation, the Allure framework works correctly for Cucumber in Java, C#, Python, Kotlin. The problem may be related to changes in Cucumber itself (precisely in js package @cucumber/cucumber) cause for old cucumber versions (e.g. 6.0.5) it works correctly (see e.g. https://github.com/allure-examples/allure-javascript-cucumberjs/blob/master/package.json). It does not work correctly for versions above 7.

DanielDelimata avatar Mar 01 '23 19:03 DanielDelimata

This is happening on all integrations of this version Also on a integration of playWright https://github.com/Tallyb/cucumber-playwright

Jordanlosa97 avatar Mar 10 '23 20:03 Jordanlosa97

Any chance this can be picked up? I upgraded today from cucumber 7 to 9 and starting having this issue. To be fair, I was using "allure-cucumberjs8": "^2.0.0-beta.15", which had its own issues but at least test were displaying correctly.

Update: I just tried to run Cucumber9 with the beta version of allure-cucumberjs8 and this is not a bug there so definitely not a cucumber issue.

ghost avatar Jul 20 '23 08:07 ghost

The same problem, it's a bug. And as a workaround, I had to add a variable to the scenario header. image

Zychkov avatar Aug 02 '23 07:08 Zychkov

Fixed in 3.0.0-beta.11

baev avatar Aug 30 '24 11:08 baev