reqnroll
Soft Assertion Failures in [AfterScenario] not marking Scenario as Failed in CMD/Test Results
Reqnroll Version
2.4.1
Which test runner are you using?
NUnit
Test Runner Version Number
2.4.1
.NET Implementation
.NET 8.0
Test Execution Method
Command line – PLEASE SPECIFY THE FULL COMMAND LINE
Content of reqnroll.json configuration file
No response
Issue Description
We are using Reqnnroll with NUnit and Selenium for our BDD test automation. Our framework uses a custom SoftAssert class to collect multiple assertion failures during a scenario and throws an exception in the [AfterScenario] hook (teardown). Problem: • When a soft assertion failure occurs, the exception is thrown in [AfterScenario]. • The failure is correctly logged in our log files (e.g., logs.txt), and screenshots are taken. • However, the test runner (CMD output) reports the scenario as "Passed" instead of "Failed". • This leads to false positives: failed tests are not visible in the test results, only in logs. Expected Behavior: • If an exception is thrown in [AfterScenario], the scenario should be marked as failed in the test runner output (CMD, test explorer, CI pipeline, etc.), just as it is when an exception is thrown during a step.
Steps to Reproduce
- Use Reqnnroll with NUnit and Selenium.
- Implement a SoftAssert class that collects failures and throws in [AfterScenario].
- Run a scenario with a failing soft assertion.
- Observe that the failure is logged and screenshot is taken, but the scenario is reported as "Passed" in CMD/test results.
Link to Repro Project
No response
I think this is an NUnit behavior. Could you please try with a simple NUnit sample project, that you throw an exception only from a TestTearDown method and see how the CMD output looks like?
I think this is an NUnit behavior. Could you please try with a simple NUnit sample project, that you throw an exception only from a
TestTearDownmethod and see how the CMD output looks like?
I tested with a plain NUnit test and confirmed that exceptions thrown in [TearDown] do fail the test in CMD and test explorer. The issue only occurs when using Reqnnroll [AfterScenario] hooks, where exceptions do not mark the scenario as failed in the runner output.
this is the output from CMD:
Failed PassingTest [24 ms] Error Message: TearDown : System.Exception : Exception in TearDown Stack Trace: at Automation.Framework.Core.WebUI.NUnitTearDownBehaviorTest.PassingTest() in D:\Blueprint\frontend-tests-blueprint\Automation.Framework.Core.WebUI\NUnitTearDownBehaviorTest.cs:line 10
--TearDown at Automation.Framework.Core.WebUI.NUnitTearDownBehaviorTest.TearDown() in D:\Blueprint\frontend-tests-blueprint\Automation.Framework.Core.WebUI\NUnitTearDownBehaviorTest.cs:line 16 at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor) at System.Reflection.MethodBaseInvoker.InvokeWithNoArgs(Object obj, BindingFlags invokeAttr)
