Allow test engine to be created inside a running test on .NET Core

[ChrisMaddock]

This is a useful feature when creating a runner, to be able to do full end-to-end tests.

There is a test for this named NUnit.Engine.Tests.Acceptance.InstantiateEngineInsideTest.CanInstantiateEngineInsideTest which is currently disabled for .NET Core.

The issue is that the TestEngineActivator is unable to be cast the engine to an ITestEngine instance - likely due to incorrect types being in scope somewhere. Unfortunately I've had to break this out of my current work as a separate issue.

[ChrisMaddock]

Just found that NUnit.Engine.Tests.WorkingDirectoryTests.EngineCanBeCreatedFromAnyWorkingDirectory is also affected. (Test was previously silently returning a null.)

[JamesChristie]

@ChrisMaddock Is there a workaround for this? I only just hit this issue today while trying to run a test that is covering another project that is defininf a custom NUnit runner via the engine api. We have some alternative on our end, but it would be nice to be able to retain this particular test case.

[CharliePoole]

The workaround we have used in the past is to provide a unique name to the TestAgency when it is created. However, this is only possible if your test is creating the engine, not the SUT.

[JamesChristie]

Thanks for the reply!

Currently we have our main project in the solution that defines our custom runner and some test cases itself for verifying some system behavior elsewhere. We have a unit test project alongside it that has an integration test case where we try and create our entry point and run a set of tests. That's where everything breaks, since we're trying to test a project that is, itself, utilizing NUnit for our purposes.

Would you be able to expand a bit on the TestAgency thing you mentioned? (That or link me to the possibly obvious documentation I may have missed. 😑 )

[CharliePoole]

Here's code where we test TestAgency itself. We construct it so that the URL includes "TestAgencyTest" rather than just "TestAgency".

https://github.com/nunit/nunit-console/blob/4a1820c4fcec38329ecb4473ccd244d6572d91a5/src/NUnitEngine/nunit.engine.tests/Services/TestAgencyTests.cs#L21

However, you want this engine to be injected in the instance of TestEngine, which is returned to you by TestEngineActivator.CreateInstance. The trick here is that your engine instance is initialized lazily. It's services are not created until you need one of them... which is when you either get an engine or reference the Services property. You can initialize the engine yourself immediately on getting the instance. Here's some code you can try. (Your assembly needs to reference nunit.engine itself in order to do this.

            // HACK! Convert interface to the class, which we happen to know.
            var context = engine.Services as ServiceContext;

            Services.Add(new SettingsService(true));
            Services.Add(new RecentFilesService());
            Services.Add(new TestFilterService());
            Services.Add(new ExtensionService());
            Services.Add(new ProjectService());
#if NETFRAMEWORK
            Services.Add(new DomainManager());
            Services.Add(new RuntimeFrameworkService());
            Services.Add(new TestAgency("SecondLevelAgency", 0));
#endif
            Services.Add(new DriverService());
            Services.Add(new ResultService());
            Services.Add(new DefaultTestRunnerFactory());

You may need to debug this since it's untested. It duplicates the initialization, which TestEngine would eventually do for you, with the exception of the name given to the second level TestAgency.

Because of the need to reference the engine and the hacky cast, I don't see it as suitable for inclusion in production code, but it's not too terrible in a test.

[JamesChristie]

Thank you so much for the detailed response! I'll mess with this a bit on my end and see if I can't get our test working again. 🙂

[ChrisMaddock]

Just to add to this - are you using the .NET Core engine @JamesChristie, or .NET Framework?

The .NET Framework version has the workaround Charlie describes. It should be fixed by #947, that will be in the next release, or you can use the pre-release version from our MyGet feed (see the README for details there)

If it's the .NET Core version of the engine, then afraid I'm not sure why that doesn't work at the moment, so don't have a workaround for it. I think it's a separate problem to the TestAgency one though, as the .NET Core version doesn't create the remoting URI which causes the failure on .NET Framework.