Reqnroll icon indicating copy to clipboard operation
Reqnroll copied to clipboard
verified publisher icon reqnroll

exception with Nunit 4

Open PolkaDan opened this issue 5 months ago • 8 comments

Reqnroll Version

latest nuget

Which test runner are you using?

NUnit

Test Runner Version Number

I don t remember

.NET Implementation

.NET 9.0

Test Execution Method

Visual Studio Test Explorer

Content of reqnroll.json configuration file

No response

Issue Description

I added a dummy scenario where steps throw exceptions (pending and others) also Assert.Fail().the runner passes the scenario correctly but before reaching AfterScenario, it gives a CLR invalid program exception. The stack trace shows NUnit calls.

I decided to switch to latest Nunit 3 to see if nunit version is involved and it worked, no exception and the test completed as expected.

I also have latest playwright in reference but I didn t use it yet

Steps to Reproduce

create a hook with Before/After feature and scenario.

create a dummy scenario with the default steps that throws a pending exception.

debug the scenario.

Link to Repro Project

No response

PolkaDan avatar Sep 05 '25 22:09 PolkaDan

Please post the full stacktrace and exception details, thanks!

304NotModified avatar Sep 06 '25 00:09 304NotModified

Log level is set to Informational (Default).

Connected to test environment '< Local Windows Environment >'
Test data store opened in 0,240 sec.
========== Starting test discovery ==========
NUnit Adapter 5.1.0.0: Test discovery starting
NUnit Adapter 5.1.0.0: Test discovery complete
========== Test discovery finished: 1 Tests found in 8,9 sec ==========
Building Test Projects
========== Starting test run ==========
NUnit Adapter 5.1.0.0: Test execution started
Running selected tests in C:\Projects\Git\STM.RSI\Main\Output\bin\UIAutomation\STM.RSI.WebApp.UIAutomation.dll
   NUnit3TestExecutor discovered 1 of 1 NUnit test cases using Current Discovery mode, Non-Explicit run
Unhandled exception. System.InvalidProgramException: Common Language Runtime detected an invalid program.
   at NUnit.Framework.Internal.TestResult.ExceptionResult..ctor(Exception ex, FailureSite site)
   at NUnit.Framework.Internal.TestResult.RecordException(Exception ex, FailureSite site)
   at NUnit.Framework.Internal.TestResult.RecordException(Exception ex)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork()
   at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread()
   at NUnit.Framework.Internal.Execution.WorkItem.Execute()
   at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(WorkItem work, ParallelExecutionStrategy strategy)
   at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(WorkItem work)
   at NUnit.Framework.Internal.Execution.CompositeWorkItem.RunChildren()
   at NUnit.Framework.Internal.Execution.CompositeWorkItem.PerformWork()
   at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread()
   at NUnit.Framework.Internal.Execution.WorkItem.Execute()
   at NUnit.Framework.Internal.Execution.TestWorker.TestWorkerThreadProc()
   at System.Threading.Thread.StartCallback()
Given une liste d'url
-> error:  (0,1s)
When chaque url est appelé
-> skipped because of previous errors
Then le résulat est celui espéré
-> skipped because of previous errors

The active test run was aborted. Reason: Test host process crashed : Unhandled exception. System.InvalidProgramException: Common Language Runtime detected an invalid program.
   at NUnit.Framework.Internal.TestResult.ExceptionResult..ctor(Exception ex, FailureSite site)
   at NUnit.Framework.Internal.TestResult.RecordException(Exception ex, FailureSite site)
   at NUnit.Framework.Internal.TestResult.RecordException(Exception ex)
   at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork()
   at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread()
   at NUnit.Framework.Internal.Execution.WorkItem.Execute()
   at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(WorkItem work, ParallelExecutionStrategy strategy)
   at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(WorkItem work)
   at NUnit.Framework.Internal.Execution.CompositeWorkItem.RunChildren()
   at NUnit.Framework.Internal.Execution.CompositeWorkItem.PerformWork()
   at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread()
   at NUnit.Framework.Internal.Execution.WorkItem.Execute()
   at NUnit.Framework.Internal.Execution.TestWorker.TestWorkerThreadProc()
   at System.Threading.Thread.StartCallback()

========== Test run aborted: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========

PolkaDan avatar Sep 08 '25 13:09 PolkaDan

Thanks!

304NotModified avatar Sep 08 '25 18:09 304NotModified

Connected to test environment '< Local Windows Environment >' Test data store opened in 0,240 sec. ========== Starting test discovery ========== NUnit Adapter 5.1.0.0: Test discovery starting NUnit Adapter 5.1.0.0: Test discovery complete ========== Test discovery finished: 1 Tests found in 8,9 sec ========== Building Test Projects ========== Starting test run ========== NUnit Adapter 5.1.0.0: Test execution started Running selected tests in C:\Projects\Git\STM.RSI\Main\Output\bin\UIAutomation\STM.RSI.WebApp.UIAutomation.dll NUnit3TestExecutor discovered 1 of 1 NUnit test cases using Current Discovery mode, Non-Explicit run Unhandled exception. System.InvalidProgramException: Common Language Runtime detected an invalid program. at NUnit.Framework.Internal.TestResult.ExceptionResult..ctor(Exception ex, FailureSite site) at NUnit.Framework.Internal.TestResult.RecordException(Exception ex, FailureSite site) at NUnit.Framework.Internal.TestResult.RecordException(Exception ex) at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork() at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread() at NUnit.Framework.Internal.Execution.WorkItem.Execute() at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(WorkItem work, ParallelExecutionStrategy strategy) at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(WorkItem work) at NUnit.Framework.Internal.Execution.CompositeWorkItem.RunChildren() at NUnit.Framework.Internal.Execution.CompositeWorkItem.PerformWork() at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread() at NUnit.Framework.Internal.Execution.WorkItem.Execute() at NUnit.Framework.Internal.Execution.TestWorker.TestWorkerThreadProc() at System.Threading.Thread.StartCallback() Given une liste d'url -> error: (0,1s) When chaque url est appelé -> skipped because of previous errors Then le résulat est celui espéré -> skipped because of previous errors

The active test run was aborted. Reason: Test host process crashed : Unhandled exception. System.InvalidProgramException: Common Language Runtime detected an invalid program. at NUnit.Framework.Internal.TestResult.ExceptionResult..ctor(Exception ex, FailureSite site) at NUnit.Framework.Internal.TestResult.RecordException(Exception ex, FailureSite site) at NUnit.Framework.Internal.TestResult.RecordException(Exception ex) at NUnit.Framework.Internal.Execution.SimpleWorkItem.PerformWork() at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread() at NUnit.Framework.Internal.Execution.WorkItem.Execute() at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(WorkItem work, ParallelExecutionStrategy strategy) at NUnit.Framework.Internal.Execution.ParallelWorkItemDispatcher.Dispatch(WorkItem work) at NUnit.Framework.Internal.Execution.CompositeWorkItem.RunChildren() at NUnit.Framework.Internal.Execution.CompositeWorkItem.PerformWork() at NUnit.Framework.Internal.Execution.WorkItem.RunOnCurrentThread() at NUnit.Framework.Internal.Execution.WorkItem.Execute() at NUnit.Framework.Internal.Execution.TestWorker.TestWorkerThreadProc() at System.Threading.Thread.StartCallback()

========== Test run aborted: 0 Tests (0 Passed, 0 Failed, 0 Skipped) run in < 1 ms ==========

On Mon, 8 Sept 2025 at 14:15, Julian Verdurmen @.***> wrote:

304NotModified left a comment (reqnroll/Reqnroll#811) https://github.com/reqnroll/Reqnroll/issues/811#issuecomment-3267394122

Thanks!

— Reply to this email directly, view it on GitHub https://github.com/reqnroll/Reqnroll/issues/811#issuecomment-3267394122, or unsubscribe https://github.com/notifications/unsubscribe-auth/BW7VOUKPKS3M6DEDDV6HMTL3RXBVLAVCNFSM6AAAAACFYNVIY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTENRXGM4TIMJSGI . You are receiving this because you authored the thread.Message ID: @.***>

PolkaDan avatar Sep 09 '25 11:09 PolkaDan

Would it be possible for you to share the project with us? Reproduction will be much easier.

If not, what exception is being thrown in the Given step? What hooks do you have defined and do any of them throw any exceptions?

clrudolphi avatar Sep 09 '25 13:09 clrudolphi

Hi, here's a striped version of my project with the bare minimum and I still get the issue. Test.WebApp.UIAutomation.zip https://drive.google.com/file/d/1X4s5UTzg9zFHX9DAU1CNxfCDnmkh8PeA/view?usp=drive_web

On Tue, 9 Sept 2025 at 09:44, Chris Rudolphi @.***> wrote:

clrudolphi left a comment (reqnroll/Reqnroll#811) https://github.com/reqnroll/Reqnroll/issues/811#issuecomment-3270817844

Would it be possible for you to share the project with us? Reproduction will be much easier.

If not, what exception is being thrown in the Given step? What hooks do you have defined and do any of them throw any exceptions?

— Reply to this email directly, view it on GitHub https://github.com/reqnroll/Reqnroll/issues/811#issuecomment-3270817844, or unsubscribe https://github.com/notifications/unsubscribe-auth/BW7VOUOJKUIKKVFXNS7WT7T3R3KVVAVCNFSM6AAAAACFYNVIY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTENZQHAYTOOBUGQ . You are receiving this because you authored the thread.Message ID: @.***>

PolkaDan avatar Sep 11 '25 13:09 PolkaDan

I downloaded your zip, unzipped it, opened it in VisualStudio 2022, compiled it and ran it. It ran without presenting the Invalid Program exception.

   Source: Permissions.Feature line 5
   Duration: 63 ms

  Message: 
One or more step definitions are not implemented yet.
  AuthentificationLoginSteps.GivenUneListeDurl()

  Standard Output: 
Given une liste d'url
-> pending: AuthentificationLoginSteps.GivenUneListeDurl(): The step definition is not implemented.
When chaque url est appelé
-> skipped because of previous errors
Then le résulat est celui espéré
-> skipped because of previous errors
One or more step definitions are not implemented yet.
  AuthentificationLoginSteps.GivenUneListeDurl()

I also ran dotnet test from the command line and observed the same.

clrudolphi avatar Sep 11 '25 17:09 clrudolphi

I get it after the scenario runs but before it reaches AfterScenario. If I understand well, this exception is not in tests output and in debug it doesn't stop after passing the scenario. Ok then, I suppose you are using windows 11 too so I guess I'll have to stay with nunit 3.14 until I find what the issue come from then.

thanks for your time.

On Thu, 11 Sept 2025 at 13:21, Chris Rudolphi @.***> wrote:

clrudolphi left a comment (reqnroll/Reqnroll#811) https://github.com/reqnroll/Reqnroll/issues/811#issuecomment-3281976121

I downloaded your zip, unzipped it, opened it in VisualStudio 2022, compiled it and ran it. It ran without presenting the Invalid Program exception.

 Source: Permissions.Feature line 5  Duration: 63 ms

Message: One or more step definitions are not implemented yet. AuthentificationLoginSteps.GivenUneListeDurl()

Standard Output: Given une liste d'url -> pending: AuthentificationLoginSteps.GivenUneListeDurl(): The step definition is not implemented. When chaque url est appelé -> skipped because of previous errors Then le résulat est celui espéré -> skipped because of previous errors One or more step definitions are not implemented yet. AuthentificationLoginSteps.GivenUneListeDurl()

I also ran dotnet test from the command line and observed the same.

— Reply to this email directly, view it on GitHub https://github.com/reqnroll/Reqnroll/issues/811#issuecomment-3281976121, or unsubscribe https://github.com/notifications/unsubscribe-auth/BW7VOUMVEO4ZNL55F47RVYT3SGVQXAVCNFSM6AAAAACFYNVIY6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTEOBRHE3TMMJSGE . You are receiving this because you authored the thread.Message ID: @.***>

PolkaDan avatar Sep 11 '25 18:09 PolkaDan

Closing this due to inactivity. Please let us know if this still an issue and please provide the requested info - thanks!

304NotModified avatar Dec 19 '25 04:12 304NotModified