Assert.Ignore in BeforeFeature hook gives System.NullReferenceException

[zsbolgar]

Reqnroll Version

2.4.0

Which test runner are you using?

NUnit

Test Runner Version Number

4.3.2

.NET Implementation

.NET Framework 4.6.2 or later

Test Execution Method

ReSharper Test Runner

Content of reqnroll.json configuration file

No response

Issue Description

Using Assert.Ignore in the AfterFeature gives error in the TearDown. In BeforeScenario it works fine.

     AddTwoNumbersThirdDataToIgnore Failed: for testing
for testing
TearDown : System.NullReferenceException : Object reference not set to an instance of an object.
   at ReqnrollTemplateProject1.Hooks.CalculatorHooks.BeforeFeatureToIgnore() in c:\Projects\ReqnrollTemplateProject1\ReqnrollTemplateProject1\Hooks\CalculatorHooks.cs:line 56
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Reqnroll.Bindings.BindingInvoker.<InvokeBindingAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.Infrastructure.TestExecutionEngine.<InvokeHookAsync>d__50.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at Reqnroll.Infrastructure.TestExecutionEngine.<FireEventsAsync>d__48.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.Infrastructure.TestExecutionEngine.<OnFeatureStartAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.TestRunner.<OnFeatureStartAsync>d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at ReqnrollTemplateProject1.Features.CalculatorFeature.<TestInitializeAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.GetResult()
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await[TResult](TestExecutionContext context, Func`1 invoke)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUpOrTearDownMethod(TestExecutionContext context, IMethodInfo method)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUp(TestExecutionContext context)
   at NUnit.Framework.Internal.Commands.BeforeAndAfterTestCommand.<>c__DisplayClass1_0.<Execute>b__0()
   at NUnit.Framework.Internal.Commands.DelegatingTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext context, Action action)

--TearDown
   at Reqnroll.Infrastructure.TestExecutionEngine.<OnScenarioEndAsync>d__39.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Reqnroll.TestRunner.<OnScenarioEndAsync>d__17.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.GetResult()
   at ReqnrollTemplateProject1.Features.CalculatorFeature.<TestTearDownAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.GetResult()
   at NUnit.Framework.Internal.AsyncToSyncAdapter.Await[TResult](TestExecutionContext context, Func`1 invoke)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunSetUpOrTearDownMethod(TestExecutionContext context, IMethodInfo method)
   at NUnit.Framework.Internal.Commands.SetUpTearDownItem.RunTearDown(TestExecutionContext context)

-> warning: The previous ScenarioContext was already disposed.
-> warning: The previous ScenarioContext was already disposed.

Steps to Reproduce

[BeforeFeature("Calculator")] public static void BeforeFeatureToIgnore() { Assert.Ignore("for testing"); }

Link to Repro Project

No response

[304NotModified]

What's the namespace of Assert?

And please post your package references.

[zsbolgar]

The namespace of Assert is: Nunit.Framework.Assert Package references:

<ItemGroup>
   <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />
   <PackageReference Include="Reqnroll.ExternalData" Version="2.4.0" />
   <PackageReference Include="Reqnroll.NUnit" Version="2.4.0" />
   <PackageReference Include="nunit" Version="4.3.2" />
   <PackageReference Include="NUnit3TestAdapter" Version="5.0.0" />
 </ItemGroup>

Can see the code here: https://github.com/zsbolgar/ReqnrollTemplateProject1 With the latest commit I inserted the Assert.Ignore

[clrudolphi]

This should be fixed by 560

[304NotModified]

This should be fixed by 560

@zsbolgar you could test that with the packages found here: https://github.com/reqnroll/Reqnroll/actions/runs/14619744429/artifacts/2994189305 (version 2.4.1-ci20250423-957)

[gasparnagy]

Hopefully fixed by v2.4.1. Please verify and reopen if not.

[FermJacob]

@gasparnagy I just upgraded to 2.4.1 and I have the same issue with Reqnroll.MSTest.

I am creating a demo project and I am getting this same null reference when I perform the below, in a new step definition file

throw new PendingStepException();

I can't grab the stacktrace currently, but it is being thrown by

SkippedStepsHandler.Handle(ScenarioContext scenarioContext)
TestExecutionEngine.OnSkipStep()

[obligaron]

@FermJacob Could you provide a sample repro?

[304NotModified]

I can confirm that the issue is reproduceable with 2.4.0 with the test AddTwoNumbers in the project ExceptionAfterFeature (not when running the test separate, but when running all the tests) - using the source of https://github.com/zsbolgar/ReqnrollTemplateProject1 (6a7744928e7cc9c1cd5906643b4093d8eea4a6c9)

I also can confirm this is fixed in 2.4.1 with the repo case:

Please also note there is a bug in the hook of ReqnrollTemplateProject1, and this will give also a nullref with 2.4.1 as _scenarioContext.ScenarioInfo.Arguments["testdata"] gives null

 [AfterScenario("DataSourceScenarioTag")]
    public void AfterDataSourceScenario()
    {
        Assert.Fail($"throw assert exception of {_scenarioContext.ScenarioInfo.Arguments["testdata"].ToString()}");
    }

So this case is fixed in 2.4.1

@FermJacob if this is still an issue with your case, please open a new issue with more details and a reproduction. Thanks!