TUnit test framework

[AdaskoTheBeAsT]

🤔 What's changed?

Added implementation of TUnit Provider

⚡️ What's your motivation?

This is completelly new feature for test framework based on source generation

🏷️ What kind of change is this?

• :zap: New feature (non-breaking change which adds new behaviour)

♻️ Anything particular you want feedback on?

I am not sure if this is proper way of creating plugin - I tried to adjust all the stuff similar to NUnit xUnit and MSTest implementation.

📋 Checklist:

• [x] I've changed the behaviour of the code
  • [ ] I have added/updated tests to cover my changes.
• [X] My change requires a change to the documentation.
  • [ ] I have updated the documentation accordingly.
• [ ] Users should know about my change
  • [ ] I have added an entry to the "[vNext]" section of the CHANGELOG, linking to this pull request & included my GitHub handle to the release contributors list.

---

This text was originally taken from the template of the Cucumber project, then edited by hand. You can modify the template here.

[thomhurst]

Will this work? Tunit uses source generators and so does reqnroll. Source generated code can't see other source generated code so it may not be able to generate test cases: https://github.com/orgs/reqnroll/discussions/418

[AdaskoTheBeAsT]

it is beginning but apparently yes

[thomhurst]

Nice work 😁

[gasparnagy]

@AdaskoTheBeAsT This looks pretty good so far. 🤘

One small comment: traditionally the unit test provider classes have been added to the UnitTestProvider folder of the generator project, but this was because at that time we did not have a plugin system. In your case, you can (and you should IMHO) put your unit test provider class to the generator plugin project and add move the registration to the plugin class like we do for example here.

(In principle we should have moved the NUnit/xUnit/MsTest stuff as well to their plugins, just we haven't done that yet.)

[AdaskoTheBeAsT]

@gasparnagy thanks for feedback I moved that registration to plugin

[gasparnagy]

@AdaskoTheBeAsT is this ready for final review?

[AdaskoTheBeAsT]

hi I will need to check this through the weekend - and then mark it is ready for review - sorry was busy recently

[snow-jallen]

This would be wonderful! Thank you for making this happen. :-) 🙏

[AdaskoTheBeAsT]

sorry it took me ultra long to get back to this - I am not sure what I can improve here - but anyway ready to feedback

[Odonno]

This looks like it's ready to be shipped. Any idea when this can be available?

[thomhurst]

Has this been reviewed? 😄

[obligaron]

Could you add a TUnitGenerationTest' under SystemTests\Generation' (see `MsTestGenerationTest' for an example). This will ensure that the basic functionality of Reqnroll is tested with TUnit in our CI/CD pipeline (we do this with all supported testing frameworks).

Thanks for the great work on adding TUnit to Reqnroll 🙂!

[AdaskoTheBeAsT]

@obligaron I tried to adjust all the tests to also use TUnit - still 37 tests are failing - no idea how to fix that it seems module initializer is not able to load TUnit.Core and I do not know why

[thomhurst]

Let me know if I can help and how to reproduce any errors if so

[obligaron]

I fixed the compilation error. Background is:

      <!-- Reqnroll.TUnit.ReqnrollPlugin is referenced only to be included in the NuGet package. -->
      <!-- Do not add it as a normal reference, as the TUnit SourceGenerator adds generated code that depends on TUnit.Core. -->

I also added trx support for TUnit for our test infrastructure. But we need the test output in the trx files. I think https://github.com/thomhurst/TUnit/issues/2218 is related.

Also some compilation errors are left when Tags are used.

Edit: It's interesting that the compilation error isn't present on my local machine. 🤔

[304NotModified]

Maybe locally it isn't compiled for all target frameworks and/or the creation of the nuget packages is different?

[AdaskoTheBeAsT]

@obligaron thanks for help - you made my day!

[thomhurst]

But we need the test output in the trx files

Should be available in the next version

[thomhurst]

Seen the branch is now updated to the latest version. Is the TRX okay now?

[AdaskoTheBeAsT]

16 errors left - and now I think I am approaching some differencies between libs - @thomhurst could you please shed some light how DisplayName is calculated when multiple Arguments are setup on tests?

in reqnroll I have

and one test is failing in reqnroll because it expects that display name will somewhat contain argument in name

[thomhurst]

16 errors left - and now I think I am approaching some differencies between libs - @thomhurst could you please shed some light how DisplayName is calculated when multiple Arguments are setup on tests?

in reqnroll I have

and one test is failing in reqnroll because it expects that display name will somewhat contain argument in name

https://github.com/thomhurst/TUnit/blob/main/TUnit.Core/Extensions/TestContextExtensions.cs#L91

Each argument attribute is registered as a separate test. And each separate test has its own separate test context.

If you can grab the test context object, you can just call the extension method yourself and tap into TUnit logic

[deastew]

Do we have any idea when this might be ready for release? We have recently started using TUnit for integration/functional tests and I would love to start using ReqnRoll

[obligaron]

I investigated why the Before_After_Feature_hooks_execute_only_once_on_sequential_run_even_with_failing_feature_hooks test fails.

This test disables parallel execution and checks whether a BeforeFeature/AfterFeature is executed only once per feature.

Background: To reduce overhead and allow users to reuse resources (e.g., browser instances), Reqnroll caches test resources (test runners/test contexts). After every scenario execution, the test runner is returned to a pool of available runners. When starting a new scenario, a runner is selected from the pool (preferably one with the same feature information). Reqnroll checks the runner for the current feature information and starts a new feature if it has not yet been opened or is different from the one that was executed.

This logic assumes that test frameworks execute all tests in a class before running the next test from the next class. Note: Features are allowed to run in parallel, but executing features out of order ("pause" or delay a started class) is not supported.

TUnit currently behaves differently. Here is a small example without Reqnroll:

[assembly: NotInParallel]

namespace TUnitProject;

public class Feature1
{
    [Before(Class)]
    public static async Task Before()
    {
        await Task.Yield();
        Console.WriteLine("Before1");
    }

    [Test]
    //[NotInParallel(Order = 1)]
    public async Task TestA()
    {
        await Task.Yield();
        Console.WriteLine("Test1A");
    }

    [Test]
    //[NotInParallel(Order = 2)]
    public async Task TestB()
    {
        await Task.Yield();
        Console.WriteLine("Test1B");
    }

    [After(Class)]
    public static async Task After()
    {
        await Task.Yield();
        Console.WriteLine("After1");
    }
}

public class Feature2
{
    [Before(Class)]
    public static async Task Before()
    {
        await Task.Yield();
        Console.WriteLine("Before2");
    }

    [Test]
    //[NotInParallel(Order = 3)]
    public async Task TestA()
    {
        await Task.Yield();
        Console.WriteLine("Test2A");
    }

    [Test]
    //[NotInParallel(Order = 4)]
    public async Task TestB()
    {
        await Task.Yield();
        Console.WriteLine("Test2B");
    }
    [After(Class)]
    public static async Task After()
    {
        await Task.Yield();
        Console.WriteLine("After2");
    }
}

public class Feature3
{
    [Before(Class)]
    public static async Task Before()
    {
        await Task.Yield();
        Console.WriteLine("Before3");
    }

    [Test]
    //[NotInParallel(Order = 5)]
    public async Task TestA()
    {
        await Task.Yield();
        Console.WriteLine("Test3A");
    }

    [Test]
    //[NotInParallel(Order = 6)]
    public async Task TestB()
    {
        await Task.Yield();
        Console.WriteLine("Test3B");
    }
    [After(Class)]
    public static async Task After()
    {
        await Task.Yield();
        Console.WriteLine("After3");
    }
}

Output:

[TUnitProject.dll] Before1 [TUnitProject.dll] Test1A [TUnitProject.dll] Before3 [TUnitProject.dll] Test3B [TUnitProject.dll] Test3A [TUnitProject.dll] After3 [TUnitProject.dll] Before2 [TUnitProject.dll] Test2B [TUnitProject.dll] Test2A [TUnitProject.dll] After2 [TUnitProject.dll] Test1B [TUnitProject.dll] After1

Ideally, TUnit should prefer in-order execution (sort by class). However, Reqnroll should also handle out-of-order execution of features.

I opened #638 to support this in Reqnroll.

[obligaron]

Do we have any idea when this might be ready for release? We have recently started using TUnit for integration/functional tests and I would love to start using ReqnRoll

Some tests ensuring that the current TUnit Plugin for Reqnroll runs correctly are still failing. These issues need to be addressed first to ensure that everything, including some rarely used features, works as expected. Hopefully, we will complete these changes in time for the V3 release in a few weeks.

[obligaron]

The test Scenario_outline_examples_gather_tags_and_parameters fails, because TUnit implicitly converts one (null) parameter to an array with one null value.

See the following TUnit example (without Reqnroll):

    [Test]
    [Arguments("sample", null)]
    public void SampleWithNull(string text, string[] tags)
    {
        if (tags is null)
            Console.WriteLine("expected");
        else
            Console.WriteLine("different?");
    }

It will print "different?" because the null parameter is generated as string[ ] methodArg1 = [null]. This was likely done to support the params keyword.

Could TUnit perhaps check explicitly for the params keyword and only convert arguments to an array in these cases? Otherwise, we would have to implement a workaround in Reqnroll. Or perhaps there is another solution?

@thomhurst Any suggestions?

[thomhurst]

The test Scenario_outline_examples_gather_tags_and_parameters fails, because TUnit implicitly converts one (null) parameter to an array with one null value.

See the following TUnit example (without Reqnroll):

    [Test]
    [Arguments("sample", null)]
    public void SampleWithNull(string text, string[] tags)
    {
        if (tags is null)
            Console.WriteLine("expected");
        else
            Console.WriteLine("different?");
    }

It will print "different?" because the null parameter is generated as string[ ] methodArg1 = [null]. This was likely done to support the params keyword.

Could TUnit perhaps check explicitly for the params keyword and only convert arguments to an array in these cases? Otherwise, we would have to implement a workaround in Reqnroll. Or perhaps there is another solution?

@thomhurst Any suggestions?

The suggestion would be for me to fix that haha. Can you raise it as an issue on the TUnit repo so I don't forget about it?

[gasparnagy]

Or perhaps there is another solution?

We could generate an empty array instead. I think the actual code would also support empty arrays as well, but needs to be checked.

[gasparnagy]

Note: Features are allowed to run in parallel, but executing features out of order ("pause" or delay a started class) is not supported.

@obligaron This is not exactly like that, or it depends on how we interpret the word "support". But I'll continue my notes at the PR to keep this one focusing on TUnit.

[obligaron]

I opened https://github.com/thomhurst/TUnit/issues/2589 for tracking the issue.

As gaspar mentioned, we can implement a workaround it if the TUnit backlog is too full.

I didn't want to make any stress, but only wanted to highlight the different behavior between TUnit and other testframeworks and TUnit and the c# compiler regarding params.

Thanks for your support tom 🙂

[thomhurst]

I'll try and get on it! Currently in a massive refactor that is beating me haha but after I beat it I'll try and get on it. It's also finding time between life and work

On Thu, 19 Jun 2025, 20:51 obligaron, @.***> wrote:

obligaron left a comment (reqnroll/Reqnroll#442) https://github.com/reqnroll/Reqnroll/pull/442#issuecomment-2989034495

I opened thomhurst/TUnit#2589 https://github.com/thomhurst/TUnit/issues/2589 for tracking the issue.

As gaspar mentioned, we can implement a workaround it if the TUnit backlog is too full.

I didn't want to make any stress, but only wanted to highlight the different behavior between TUnit and other testframeworks and TUnit and the c# compiler regarding params.

Thanks for your support tom 🙂

— Reply to this email directly, view it on GitHub https://github.com/reqnroll/Reqnroll/pull/442#issuecomment-2989034495, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHIROK4TBQA7MJGVZ2G6BOL3EMIETAVCNFSM6AAAAABW3OFZVOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOBZGAZTINBZGU . You are receiving this because you were mentioned.Message ID: @.***>