addNonParallelizableMarkerForTags breaks step discovery in VS 2019 SpecFlow extension

[patrykbuzowicz]

SpecFlow Version

3.9.22 (+ VS 2019 16.11.3, VS SpecFlow extension 2019.0.96.46294)

Which test runner are you using?

NUnit

Test Runner Version Number

3.13.2

.NET Implementation

equal or greater .NET Framework 4.6.1

Project Format of the SpecFlow project

Sdk-style project format

.feature.cs files are generated using

SpecFlow.Tools.MsBuild.Generation NuGet package

Test Execution Method

Visual Studio Test Explorer

SpecFlow Section in app.config or content of specflow.json

<specFlow>
  <stepAssemblies>
    <stepAssembly assembly="XXX" />
    <stepAssembly assembly="YYY" />
  </stepAssemblies>
  <generator>
    <addNonParallelizableMarkerForTags>
      <tag value="notParallel" />
    </addNonParallelizableMarkerForTags>
  </generator>
</specFlow>

Issue Description

When <addNonParallelizableMarkerForTags> is in use, SpecFlow fails to map steps in .feature files to their C# definitions. Exception thrown as captured in the Output>SpecFlow pane:

[15:45:49.0473675] VsProjectScope: Configuration loading error: System.Configuration.ConfigurationErrorsException: Unrecognized element 'addNonParallelizableMarkerForTags'.
   at System.Configuration.ConfigurationElement.DeserializeElement(XmlReader reader, Boolean serializeCollectionKey)
   at System.Configuration.ConfigurationSection.DeserializeSection(XmlReader reader)
   at TechTalk.SpecFlow.Configuration.ConfigurationSectionHandler.CreateFromXml(String xmlContent)
   at TechTalk.SpecFlow.IdeIntegration.Configuration.ConfigurationLoader.Load(SpecFlowConfiguration specFlowConfiguration, IConfigurationHolder configurationHolder)
   at TechTalk.SpecFlow.VsIntegration.Implementation.LanguageService.VsProjectScope.LoadSpecFlowConfiguration(). Using default configuration instead.

Steps to Reproduce

• prepare a SpecFlow project with some steps and features
• configure <addNonParallelizableMarkerForTags> in XML config file
• using VS+SpecFlow extension, navigate to a feature file

Expected: all steps are recognized and can navigate to their definition Actual: some steps aren't recognized and the extension offers creating them when trying to navigate.

Link to Repro Project

No response

[SabotageAndi]

If you switch to specflow.json it should work again.

[patrykbuzowicz]

Hi @SabotageAndi this seems to have helped.

Any plans to update the XML configuration schema in the VS extension? Is it just outdated there or needs more actions?

[johnathan-s]

@patrykbuzowicz Hello. I am using the .json as @SabotageAndi advised. The addNonParallelizableMarkerForTags flag appears to be ignored. The [assembly: Parallelizable(ParallelScope.Fixtures)] is set in nUnit as prescibed.
The flag is set in specflow.json with @tag1 identified in the generator json branch. Yet, the tests still execute in parallel. Is there any insights?

[SabotageAndi]

You don't add the @ in the config. Unit test for it is here: https://github.com/SpecFlowOSS/SpecFlow/blob/21c2d00a47c639c784d48985c1bf214c383cb90b/Tests/TechTalk.SpecFlow.RuntimeTests/Configuration/JsonConfigTests.cs#L377

[johnathan-s]

@SabotageAndi Thanks for getting back to me.

As the type is string[], I tried with "generator" : { "addNonParallelizableMarkerForTags" : "tag" } and "generator" : { ["addNonParallelizableMarkerForTags" : "tag"] }

Per the unit test (thanks for the link), the config looks like it is getting picked up, but the config flag does not appear to override the nunit flag [assembly: Parallelizable(ParallelScope:Fixtures)]

The objective I am trying to achieve is to allow setting the tests to run in parallel or serial with the specflow.json config. For example, if I have feature file A and feature file B with tag Aa and Bb (, set above the 'Feature' in the feature file,) respectively, I am hoping by setting addNonParallelizableMarkerForTags with value of of Aa with the outcome of test A and B run in serial. Please let me know if I am misunderstanding the documentation or implementing incorrectly.

[momovuk]

@johnathan-s: This worked for me for non-parallel execution:

{
  "generator": {
    "addNonParallelizableMarkerForTags": [ "notParallel" ]
  }
}

[arvinthseran]

@johnathan-s: The config mentioned by @momovuk worked for me. For information I am using netcoreapp3.1, SpecFlow -Version 3.9.52 and the tag has to be at the feature level ex :

@notParallel
Feature: FeatureName

not at the scenario level