nunit
Modular Structure of the Engine
@nunit/engine-team @nunit/core-team
We already have a number of individual issues around changing the engine's modular structure. I noticed that we don't have a single issue that incorporates an overview, so here it is. It's an Epic and some of it's individual parts are Epics in themselves. I've also labeled it as a Design Decision since should discuss and refine the plan.
Current Structure
Primary Modules
* nunit.engine
* nunit-agent-net462, nunit-agent-net80, etc.
Support Modules
* nunit.engine.core
* nunit.engine.api
* TestCentric.Metadata
Proposed Structure
Primary Modules
* nunit.engine
* nunit-agent-net462, nunit-agent-net80, etc. (now PLUGGABLE extensions)
|
>----nunit.agent.core
Support Modules
nunit.engine.api
NUnit.Extensibility
nunit.common
TestCentric.Metadata
Details
Engine
The engine is now dependent on nunit.engine.core, as are the agents. This creates an inter-dependency, which has frequently interfered with our ability to make changes. Any implementation code needed by the engine belongs in the engine. Issue #1578 deals with this.
Agents
As agents are becoming pluggable (see #909), they will need to be moved into individual projects and packaged separately. The engine and console runner will incorporate agents for .NET 4.6.2, 8.0 and 9.0 as dependencies. Depending on how soon we implement this, we may want to re-create the dropped .NET 6.0 agent as an optionally installable agent. In fact, we could, if desired, restore any of our past out-of-date agents as pluggable agents fairly easily, or we could support someone else who wanted to do so.
Agent Core
The nunit.agent.core assembly, also covered in #1578, will hold common code used by all our agents. We have the choice to either package it separately or use the source as a submodule. If separately packaged, it may either be referenced as a dependency by each agent package or used as a development-only dependency with the module distributed. There are advantages to each approach, which we should discuss in #909.
Extensibility
The new NUnit.Extensibility assembly will incorporate ExtensionManager and related classses in a separate package. ExtensionManager gives lower level access to extensions and is used by the engine's ExtensionService. The purpose of this split is to permit other modules than the engine to incorporate extensibility. Initially, the agents will make use of this, allowing extensions like the NUnit V2 Framework Driver to function. Eventually, we may use it for the console runner and it could even be used for NUnitLite, in order to avoid duplication of extension code. See issue #1049.
Metadata
At it's origins, NUnit used reflection to analyze assemblies before trying to load and run them. With NUnit 3, we switched to Mono.Cecil but eventually ran into problems for two reasons:
- Some users were testing code, which also used
Mono.Cecil, sometimes a different version from ours. - The package eliminated support for platforms we still wanted to support.
Facing the same problems with my TestCentric GUI, I created TestCentric.Metadata with some differences from the original:
- It has a subset of the capability of
Mono.Cecil, supporting only reading of assemblies. This makes it a lot smaller. - I retained support for all platforms we supported. It is currently built to target .NET Framework 2.0 and .NET Standard 2.0.
- Subsequently, I have made a few more changes. In particular, I changed the namespace to avoid collisions with user code when running under the .NET Framework. The package is currently at version 3.0.3.
For the present, I'm planning to keep assembly as is, only making changes as needed. The package is currently used by the ExtensionManager as well as by several engine services, so it will be referenced by nunit.engine, nunit.agent.core and NUnit.Extensibility.
NUnit Common
We currently provide internal trace facilities to our various modules by duplicating the source code. Providing logging to several new modules would increase the number of duplicate copies. I propose to create a package ~~NUnit.InternalTrace~~ nunit.common to centralize this facility as well as other commonly used classes. In addition to eliminating code duplication, this might eventually help us combine multiple log files into a single file. I'll create a new issue for this one.
Engine Api
In issue #770, @ChrisMaddock proposed splitting the then-existing api assembly into multiple assemblies. In my plan for that issue, I didn't include that aspect of his proposal. I think should discuss it in the context of this overall issue. See further discussion below.
Thanks for the extensive write up and organization of issues here @CharliePoole . Lots to read and catch up on past discussions
This continues the initial proposal, to deal with the engine API.
Background
In #770, @ChrisMaddock proposed as follows..
...I currently see the API assembly as serving two distinct purposes.
- The API for runners wishing to reference the engine
- The assembly to be referenced by engine extensions
Based on the fact that driver extensions currently need to be loaded by agents, there is a potential need for extensions to be able to target legacy platforms, and thus the API assembly as well. This is a restriction that's only necessary for "purpose 2" however - so I wonder if there's value in splitting the API assembly? This would give us e.g.:
- nunit.engine.api.dll: Contains all the code in the current API assembly, targets the same platform(s) as nunit.engine.dll
- nunit.extensions.api.dll: A new, super-slim assembly, which contains only the interfaces required by extension developers (IProjectLoader, IResultWriter etc.). This assembly could continue to target the lowest platform we see fit - potentially .NET 2.0.
In general, I like this aproach. I think we have to consider...
- Who defines an interface... e.g. the engine has to define what runners are able to call, what extensions are available, etc.
- Who implements the interface.
- Who uses the interface. That may be runners, the engine itself, extensions, agents, etc.
- Frequency of update. It's not convenient if we mix things that are updated at different times in the same assembly.
Proposal
NUnit.Extensibility.Api
Provides the basic components of NUnit extensibility, including attributes that indicate extensions and the methods and properties avaiable to access them. Does not include any specific extensions. It is defined and implemented in the NUnit.Extensibility package and used by the engine, runners, agents and any other module wanting to make use of the capability.
As of February, 2025, working toward the first beta release, this is complete.
NUnit.Engine.Api
Currently contains everything that was not moved to the extensibilty API plus a few additions. In total, 32 different interfaces, enums and classes are defined, which seems a bit more than should be needed in the API.
Much of this is low-hanging fruit. For example, interfaces formerly implemented by engine, now implemented elsewhere or not used at all. I plan to do separate PRs for groups of these.
Additional Apis
STILL UNDER CONSTRUCTION
Listing all the parts of this Epic in one place so we can decide when it's done!
Pre-existing issues:
- [x] #909
- [x] #1578
- [x] #1049
Stuff still to do for the second beta release
- [ ] Make decision regarding #1631 and implement it.
- [x] Create NUnit.Common package and modify other packages so they reference it.
- [ ] Create NUnit.Extensibility package and modify other packages to reference it.