TUnit icon indicating copy to clipboard operation
TUnit copied to clipboard

`[Before]`, `[After]` hooks swallow the exception and fail silently

Open robertcoltheart opened this issue 5 months ago • 8 comments

Note: This applies to [After] hook as well, so I suspect all the hooks are affected?

Using the below:

public class Class1
{
    [Before(Test)]
    public void Before()
    {
        throw new ArgumentException("Bad Bad Leroy Brown");
    }

    [Test]
    public void TestMethod()
    {
    }
}

The logs yield the following in 0.61.58:

Message: 
BeforeTest hook failed

  Stack Trace: 
HookExecutor.ExecuteBeforeTestHooksAsync(AbstractExecutableTest test, CancellationToken cancellationToken)
TestExecutor.ExecuteAsync(AbstractExecutableTest executableTest, CancellationToken cancellationToken)
TestExecutor.ExecuteAsync(AbstractExecutableTest executableTest, CancellationToken cancellationToken)
TestExecutor.ExecuteAsync(AbstractExecutableTest executableTest, CancellationToken cancellationToken)
<<ExecuteTestInternalAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
RetryHelper.ExecuteWithRetry(TestContext testContext, Func`1 action)
RetryHelper.ExecuteWithRetry(TestContext testContext, Func`1 action)
TestCoordinator.ExecuteTestInternalAsync(AbstractExecutableTest test, CancellationToken cancellationToken)

This was working as expected in 0.57.24, so something has changed between then and now.

 Message: 
Bad Bad Leroy Brown

  Stack Trace: 
Class1.Before() line 8
<>c__DisplayClass2_0.<global_ClassLibrary2_Class1_Before_0Params_Body>b__0() line 80
GeneratedHookRegistry.global_ClassLibrary2_Class1_Before_0Params_Body(Object instance, TestContext context, CancellationToken cancellationToken) line 80
<<CreateTimeoutHookAction>b__1>d.MoveNext()
--- End of stack trace from previous location ---
<<CreateInstanceHookDelegateAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
SingleTestExecutor.ExecuteBeforeTestHooksAsync(IReadOnlyList`1 hooks, TestContext context, CancellationToken cancellationToken)
SingleTestExecutor.ExecuteBeforeTestHooksAsync(IReadOnlyList`1 hooks, TestContext context, CancellationToken cancellationToken)
SingleTestExecutor.ExecuteTestWithHooksAsync(AbstractExecutableTest test, Object instance, CancellationToken cancellationToken)
SingleTestExecutor.ExecuteTestWithHooksAsync(AbstractExecutableTest test, Object instance, CancellationToken cancellationToken)
SingleTestExecutor.ExecuteTestInternalAsync(AbstractExecutableTest test, CancellationToken cancellationToken)

robertcoltheart avatar Sep 30 '25 23:09 robertcoltheart

@robertcoltheart Try v0.63

thomhurst avatar Oct 02 '25 15:10 thomhurst

No change, this is the output with 0.63.3:

Message: 
BeforeTest hook failed

  Stack Trace: 
HookExecutor.ExecuteBeforeTestHooksAsync(AbstractExecutableTest test, CancellationToken cancellationToken)
TestExecutor.ExecuteAsync(AbstractExecutableTest executableTest, CancellationToken cancellationToken)
TestExecutor.ExecuteAsync(AbstractExecutableTest executableTest, CancellationToken cancellationToken)
TestExecutor.ExecuteAsync(AbstractExecutableTest executableTest, CancellationToken cancellationToken)
<<ExecuteTestInternalAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
<<ExecuteTestInternalAsync>b__0>d.MoveNext()
--- End of stack trace from previous location ---
RetryHelper.ExecuteWithRetry(TestContext testContext, Func`1 action)
RetryHelper.ExecuteWithRetry(TestContext testContext, Func`1 action)
TestCoordinator.ExecuteTestInternalAsync(AbstractExecutableTest test, CancellationToken cancellationToken)

robertcoltheart avatar Oct 02 '25 22:10 robertcoltheart

Are you talking about swallowing the inner exception messages?

thomhurst avatar Oct 02 '25 22:10 thomhurst

Correct, in earlier versions of TUnit, the actual reason/exception was shown. See:

Message: 
Bad Bad Leroy Brown

  Stack Trace: 
Class1.Before() line 8
<>c__DisplayClass2_0.<global_ClassLibrary2_Class1_Before_0Params_Body>b__0() line 80
GeneratedHookRegistry.global_ClassLibrary2_Class1_Before_0Params_Body(Object instance, TestContext context, CancellationToken cancellationToken) line 80
<<CreateTimeoutHookAction>b__1>d.MoveNext()
--- End of stack trace from previous location ---
<<CreateInstanceHookDelegateAsync>b__0>d.MoveNext()

robertcoltheart avatar Oct 02 '25 22:10 robertcoltheart

Is this just in an IDE or via dotnet run too?

thomhurst avatar Oct 02 '25 22:10 thomhurst

dotnet run works, but both VS and dotnet test don't show the inner exception.

robertcoltheart avatar Oct 02 '25 23:10 robertcoltheart

I think this is an issue with them since I'm passing through the inner exception. The dotnet test experience should be improved in .net 10 I believe.

thomhurst avatar Oct 02 '25 23:10 thomhurst

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

github-actions[bot] avatar Nov 27 '25 00:11 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Dec 02 '25 00:12 github-actions[bot]

@thomhurst Can you please reopen this? This is still an issue in .net 10 under Visual studio 2026.

For reference, using NUnit's [SetUp] feature correctly shows exceptions in the VS test output, so it leads me to think this is an issue with TUnit and not with the framework / IDE.

robertcoltheart avatar Dec 02 '25 21:12 robertcoltheart

@robertcoltheart can you compare like for like and run NUnit in Microsoft Testing Platform mode

thomhurst avatar Dec 02 '25 21:12 thomhurst

NUnit:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <EnableNUnitRunner>true</EnableNUnitRunner>
    <TargetFramework>net10.0</TargetFramework>
    <TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="18.0.1" />
    <PackageReference Include="NUnit" Version="4.4.0" />
    <PackageReference Include="NUnit3TestAdapter" Version="5.2.0" />
  </ItemGroup>

</Project>
using NUnit.Framework;
using System;

namespace NUnitProject;

[TestFixture]
public class NUnitTests
{
    [SetUp]
    public void Setup()
    {
        throw new InvalidOperationException("Bad bad");
    }

    [Test]
    public void NUnitTest()
    {
        Assert.That(true, Is.True);
    }
}
Image

TUnit:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="TUnit" Version="1.3.25" />
  </ItemGroup>

</Project>
using System;
using System.Threading.Tasks;

namespace TUnitProject;

public class TUnitTests
{
    [Before(Test)]
    public Task Setup()
    {
        throw new InvalidOperationException("Bad bad");
    }

    [Test]
    public async Task TUnitTest()
    {
        await Assert.That(true).IsTrue();
    }
}
Image

robertcoltheart avatar Dec 04 '25 09:12 robertcoltheart

Here is a solution that I used to reproduce it: Reproduce.zip

robertcoltheart avatar Dec 04 '25 09:12 robertcoltheart

Maybe the issue is that TUnit is wrapping the real exception inside a hook exception, and VS only shows the message property, not the full exception ToString. Is there a reason to wrap the exception? Why not just let the original exception throw?

robertcoltheart avatar Dec 04 '25 19:12 robertcoltheart

Maybe the issue is that TUnit is wrapping the real exception inside a hook exception, and VS only shows the message property, not the full exception ToString. Is there a reason to wrap the exception? Why not just let the original exception throw?

I think that is the problem. And the wrapping was supposed to help narrow down when the exception occurred

thomhurst avatar Dec 04 '25 19:12 thomhurst