reqnroll
With allowRowTests and Scenario outline scenario order is not correct
Reqnroll Version
3.0.2
Which test runner are you using?
MSTest
Test Runner Version Number
4
.NET Implementation
.NET Framework 4.6.2 or later
Test Execution Method
Visual Studio Test Explorer
Content of reqnroll.json configuration file
{ "$schema": "https://schemas.reqnroll.net/reqnroll-config-latest.json",
"bindingAssemblies": [ ], "runtime": { "stopAtFirstError": true }, "generator": { "allowDebugGeneratedFiles": false, "allowRowTests": false }, "formatters": { "html": {}, "message": {} } }
Issue Description
I have several Scenarios in my feature file. First scenario is to start application. After that I have scenario outline to check menu with datatabel of menu items. In .testrunsettings I set up <OrderTestsByNameInClass>false</OrderTestsByNameInClass> To preserve order of cases from my feature file If I set up "allowRowTests": true case are running in correct order, but if I set up "allowRowTests": false case order is not correct and checking of menus goes before application start In Attachement is .feature file and trx file from test execution. It is visible that in trx file first test case is not same as first case in feature file
Steps to Reproduce
In description all steps are described
Link to Repro Project
No response
In all cucumber-based testing frameworks, Scenarios are always independent of each other. Tests should not rely on execution order in any way.
If application SUT setup is needed before test execution, include that in one of the available BEFORExxx hooks.
In all cucumber-based testing frameworks, Scenarios are always independent of each other. Tests should not rely on execution order in any way.
This is understandable, but not convinient. In my particular case I have scenario with more then 100 steps. Some of them are same for other features. It is not possible to have library of steps to call from. So I managed to organize steps in scenarios. Such scenarios are called like common steps. They are same in all feature files. And they should be run in specific order. To make this order fixed I set up OrderTestByNameInClass as false and everything works correctly. The only problem is for Scenario outline with option "allowRowTests": false. I don't know why is this such problem with this settings so it creates wrong call order.
Unfortunately, what you're describing isn't the use-case for Gherkin. Each scenario is intended to be a stand-alone example of some behaviour a system must follow. The choice of using Gherkin is meant to make it easier to map your requirements to tests.
Reqnroll is designed to meet the goals and intended use of the Gherkin language, and although it might look like you should be able to use it as an automation scripting language, that's not something it's intended to cater for.
