NUnit.Engine 3.18.3 value of Attribute static property is null

[trysetnull]

There is a regression in the NUnit.Engine version 3.18.3 which we use as an NuGet dependency.

To help demonstrate the issue I have created an example project here on GitHub: trysetnull/nunit-engine-bug

The steps are as follows:

1. We set a static property on an attribute, which is used to annotate a test fixture.
2. The TestEngine is initialized and a DLL, which contains the text fixture is loaded.
3. When the tests run the value of the static property is null.

With NUnit.Engine versions 3.18.1 and 3.18.2, at point 3 the value of the static property is what has been set at step 1.

The example project has a README with more information.

[CharliePoole]

The problem here is that the static value in the framework is not the same value set by your runner program. Statics have individual values in each AppDomain and that behavior seems to persist to an extent under .NET Core.

To demonstrate this behavior, you can initialize the static property at it's point of declaration, so...

    public static string? StaticProperty { get; set; } = ExpectedValue;

You appear to be attempting to overcome a long-standing design feature (limitation?) of the nunit framework. Defining and loading of tests takes place before they are executed and that definition may not be subsequently changed.

I think we have a bit of an X/Y problem here. If you could formulate a question that relates to what you are trying to accomplish, we may be able to offer other suggestions.

[trysetnull]

Thanks for your reply but according to the docs .NET Core has exactly one AppDomain only:

On .NET Core, the AppDomain implementation is limited by design and does not provide isolation, unloading, or security boundaries. For .NET Core, there is exactly one AppDomain. Isolation and unloading are provided through AssemblyLoadContext. Security boundaries should be provided by process boundaries and appropriate remoting techniques.

https://learn.microsoft.com/en-us/dotnet/api/system.appdomain?view=net-8.0#remarks

Furthermore, given what you've said, I don't understand why the test runs successfully with version 3.18.1 and 3.18.2.

To drill down, I added the following:

var fileInfo = new FileInfo(args[0]);

// Load into the default AssemblyLoadContext.
var assembly = AssemblyLoadContext.Default.LoadFromAssemblyPath(fileInfo.FullName);
var testType = assembly.GetType("AspectService.Tests.Tests");
var attribute = testType!.GetCustomAttribute<NUnitAspectAttribute>();
var attributeType = attribute!.GetType();

if (attributeType != null)
{
    var staticProperty = attributeType.GetProperty("StaticProperty", BindingFlags.Public | BindingFlags.Static);
    if (staticProperty != null)
    {
        var staticPropertyValue = staticProperty.GetValue(null);
        Console.WriteLine($"NUnitAspectAttribute.StaticProperty: [{staticPropertyValue}]");
    }
    else
    {
        Console.WriteLine("StaticProperty not found on NUnitAspectAttribute.");
    }
}
else
{
    Console.WriteLine("NUnitAspectAttribute type not found in the assembly.");
}

using var engine = new TestEngine();
engine.WorkDirectory = fileInfo.DirectoryName;
engine.InternalTraceLevel = InternalTraceLevel.Debug;
engine.Initialize();

This outputs (as I would expect, interestingly also for version 3.18.3):

NUnitAspectAttribute.StaticProperty: [Value used for StaticProperty]

To be pedantic I even added a test that checks that the NUnit and Aspect* assemblies are in the same default assembly load context.

So I don't understand why NUnit can't "see" the current value of the static property.

It's all rather odd.

Perhaps trying to explain why I'm doing this might help to find a solution; so here goes:

We have an ASP.NET application that loads plugins; these plugins provide a REST API. Registration of various dependencies uses the built-in service container provided by .NET. Now using an integration test we want to test endpoints of the plugins, depending on the test we need a handle to the DI container. There is a host test instance that has the DI container instance and this sets the static property on the attribute (in a similar fashion to the toy code I provided). As already mentioned this works a charm up until 3.18.3.

Does that make the issue clearer? Do you have any more thoughts why it has stopped working in 3.18.3? Thanks for your time and help in advance!

Environment

Docker container running in WSL.

.NET SDK: Version: 8.0.403 Commit: c64aa40a71 Workload version: 8.0.400-manifests.18f19b92 MSBuild version: 17.11.9+a69bbaaf5

Runtime Environment: OS Name: debian OS Version: 12 OS Platform: Linux RID: linux-x64 Base Path: /usr/share/dotnet/sdk/8.0.403/

[CharliePoole]

I'm re-opening to examine the question of what changed between releases. Please let me know exactly what engine package you installed for both 3.18.2 and 3.18.3. There are, in fact some differences in what you get from various packages, which I hope to eliminate in 3.19.

WRT AppDomain under .NET Core, I'm familiar with what Microsoft says about them but skepticism about MS published docs has paid off for me in the past. In this case, I ask "One appdomain per what?" Process? Planet? Something in between? :-)

Your extra test is far from pedantic and tells us a lot. Thanks for explaining the use case.

[trysetnull]

Thanks @CharliePoole, here's the requested package versions of nunit.engine (installed from api.nuget.org) with their respective checksums:

vscode ➜ /workspaces/nunit-bug-main (main) $ dotnet nuget list source
Registered Sources:
  1.  nuget.org [Enabled]
      https://api.nuget.org/v3/index.json

# Version 3.18.1
cat /home/vscode/.nuget/packages/nunit.engine/3.18.1/nunit.engine.3.18.1.nupkg.sha512
lbcB1e8CFTAruddVX+uISZui271uNUFEta4zSG4hqaVY8X818j72+9V6vt7Ov/HUVEMSzjCwq6yChOMkZNfuJQ==

# Version 3.18.2
cat /home/vscode/.nuget/packages/nunit.engine/3.18.2/nunit.engine.3.18.2.nupkg.sha512
8fKJg0nCUkVNyydw8B3KgAZaW7XwzrfEV3x+ETPo3J28pqWR+jxq/V0BGBJBt6clNQV3N9qa4Jq9MYxcU8Z9aQ==

# Version 3.18.3
cat /home/vscode/.nuget/packages/nunit.engine/3.18.3/nunit.engine.3.18.3.nupkg.sha512 && echo 
RfGMKl1Jwcb+A7qhG74/yJ1Xry3F9FdRrYnkkoCCegiUNUhGvBWjri/KVO6qKzDx3z+PQz+D78++QQww3zQvjg==