_unitTestRuntimeProvider.TestIgnore sets the step as failed, witch presents as error in the Livingdoc report

[renycorreia]

SpecFlow Version

3.9.74

Which test runner are you using?

xUnit

Test Runner Version Number

3.9.74

.NET Implementation

.NET 6.0

Project Format of the SpecFlow project

Sdk-style project format

.feature.cs files are generated using

SpecFlow.Tools.MsBuild.Generation NuGet package

Test Execution Method

Visual Studio Test Explorer

SpecFlow Section in app.config or content of specflow.json

 {
  "bindingCulture": {
    "name": "pt-br"
  },
  "language": {
    "feature": "pt-br"
  },
  "stepAssemblies": [
    {
      "assembly": "projectName.crosscutting.ioc"
    }
  ],
  "livingDocGenerator": {
    "enabled": true,
    "filePath": "TestExecution.json"
  }
}

Issue Description

.NET Implementation

.NET 7.0 right now, but I had the same issue using .NET 6.0

SpecFlow+ LivingDoc Version

3.9.57
I've configured some tests to be ignored, based a few conditions, using _unitTestRuntimeProvider.TestIgnore("some message") in one of the steps. It works fine, except that the step and the scenario were set as failed.

I was able to change the scenario state to Skipped using the code below at the AfterScenario hook.

    if (_scenarioContext.TestError != null && _scenarioContext.TestError.ToString().Contains("[some message]"))
    {
        PropertyInfo testStatusProperty = (typeof(ScenarioContext)).GetProperty("ScenarioExecutionStatus", BindingFlags.Instance | BindingFlags.Public);
        PropertyInfo testErrorProperty = typeof(ScenarioContext).GetProperty("TestError", BindingFlags.Instance | BindingFlags.Public);

        testStatusProperty.SetValue(_scenarioContext, ScenarioExecutionStatus.Skipped);
        testErrorProperty.SetValue(_scenarioContext, null);
    }

However, the step is taken as as failed, as can be seen in the json snippet below .

This causes the Analytics tab of the Livingdoc report to indicate that there is a failed step. Even though there is no failed scenario or feature.

Is there a way of doing this without indicating that the step is failed?

Steps to Reproduce

• Skip a scenario as indicated here https://github.com/SpecFlowOSS/SpecFlow/edit/master/docs/Execution/SkippingScenarios.md
• Change the scenario state as shown in the Issue Description section
• Run tests (this generates json file)
• Generate Livingdoc.html file

Link to Repro Project

No response

[Myvryn]

I've been seeing the same issue. It would be nice to get this fixed so that we can have Inconclusive rather than failed.

[Moshex]

Same, it would be nice if the inhouse test ignore did not show as a fail on the report

[JDunn3]

This is a major issue for anyone with >10 tests in various feature files that they must dynamically ignore.