MSTEST0016 and base test class

[avivanoff]

We have the following structure in some of our test projects:

[TestClass]
public abstract class TestBase
{
    public TestContext TestContext { get; set; }

    [TestInitialize]
    public void RunTestSetup()
    {
        ...
    }

    [ClassCleanup]
    public static void ClassCleanup()
    {
        ...
    }
}

[TestClass]
public sealed class SomeTest : TestBase
{
    ...
}

After upgrading to 3.3.1 we started getting the error:

error MSTEST0016: Test class 'TestBase' should have at least one test method or be 'static' with method(s) marked by '[AssemblyInitialization]' and/or '[AssemblyCleanup]'

Should MSTEST0016 be checking for this pattern and not trigger the warning/error, or is there a better approach?

AB#2079035

[Evangelink]

Hi @avivanoff,

As usual thank you for testing our analyzers and for your continuous feedback to improve MSTest :)

Assuming you don't have test methods in TestBase then you don't need the [TestClass] attribute on the base class.

Side note not related to MSTEST0016 but [ClassCleanup] is working in a counter intuitive way (I plan to fix this in v4 as it implies breaking changes). By default it is equivalent to [AssemblyCleanup] and runs after all tests have been executed and not all tests of current class or children classes. To change this you will need to pass ClassCleanupBehavior.EndOfClass

[avivanoff]

Just to be sure. Does it mean that when you process classes marked with TestClassAttribute you include the whole hierarchy? Will DeploymentItemAttribute on TestBase be processed as well?

[avivanoff]

Assuming you don't have test methods in TestBase then you don't need the [TestClass] attribute on the base class.

Removal of TestClass from TestBase triggers MSTEST0004: Public types should be test classes.

[Evangelink]

Just to be sure. Does it mean that when you process classes marked with TestClassAttribute you include the whole hierarchy?

As far as I can see yes, we consider all base types methods.

Will DeploymentItemAttribute on TestBase be processed as well?

From the simple test I did this is working.

Removal of TestClass from TestBase triggers MSTEST0004: Public types should be test classes.

Let me discuss with the team but I think this is a false positive.

[avivanoff]

@Evangelink, any updates?

[Evangelink]

Hey @avivanoff, sorry for the missing updates here. We had a few team chat and we haven't yet decided the direction we want to take that's why I postponed the fix for the v3.5 that is happening this sprint.

[Evangelink]

@avivanoff we will be working on the fix on that this week, sorry it took so much time.

I have some side question, is TestBase defined in a different assembly? Can't it be made internal?

The more I look at MSTEST0004 the more I am thinking the rule should ignore abstract and static types.

[avivanoff]

@Evangelink, making TestBase internal would force all derived classes to be internal as well.