Align method filtering behavior between JUnit and TestNG

[pdolif]

Closes #3225 This PR aligns the method filtering behavior between JUnit and TestNG by adjusting the TestNG MethodSelector.

Problem

Given the following test classes:

public class SubClass extends SuperClass {
    @Test
    public void testInSubClass() {}
}

public abstract class SuperClass {
    @Test
    public void testInSuperClass() {}
}

public class OtherClass {
    @Test
    public void testIncluded() {}

    @Test
    public void testNotIncluded() {}
}

When using an includesFile containing SubClass only, testInSubClass and testInSuperClass are executed (as expected).

If the includesFile does not only contain class filters but also method filters, the behavior is different (with TestNG only), and testInSuperClass is not executed anymore. Example includesFile:

SubClass
OtherClass#testIncluded

With this, only testInSubClass and OtherClass#testIncluded are executed.

Root Cause

TestNG uses the MethodSelector for filtering the tests. For testInSuperClass() the MethodSelector is invoked with an ITestNGMethod instance like this:

• getRealClass() returns SuperClass
• getTestClass().getRealClass() returns SubClass

The current MethodSelector implementation uses getRealClass() and then checks whether to run SuperClass#testInSuperClass. If we have SubClass in the includesFile, the test will not be executed, even though SubClass inherits it. This is different when using JUnit instead of TestNG.

Fix

This PR changes the MethodSelector implementation to use getTestClass().getRealClass() to check whether to run a test. The behavior changes as follows:

• In the example above, testInSuperClass gets executed too.
• With SubClass#testInSuperClass in the includesFile, prior to this PR no test was executed. Now SubClass#testInSuperClass gets executed.
• With SubClass in the includesFile and SuperClass#testInSuperClass in the excludesFile, prior to this PR only SubClass#testInSubClass got executed. Now the inherited test SubClass#testInSuperClass gets executed too.

The PR includes tests that are executed with TestNG and JUnit to ensure the behavior aligns.

---

Following this checklist to help us incorporate your contribution quickly and easily:

• [x] Each commit in the pull request should have a meaningful subject line and body.
• [x] Write a pull request description that is detailed enough to understand what the pull request does, how, and why.
• [x] Run mvn clean install to make sure basic checks pass. A more thorough check will be performed on your pull request automatically.
• [x] You have run the integration tests successfully (mvn -Prun-its clean install).

If your pull request is about ~20 lines of code you don't need to sign an Individual Contributor License Agreement if you are unsure please ask on the developers list.

To make clear that you license your contribution under the Apache License Version 2.0, January 2004 you have to acknowledge this by using the following check-box.

• [x] I hereby declare this contribution to be licenced under the Apache License Version 2.0, January 2004

• [x] In any other case, please file an Apache Individual Contributor License Agreement.

[olamy]

Could you please check with this PR https://github.com/apache/maven-surefire/pull/3179 as the big refactoring might land very soon now

[pdolif]

@olamy The integration test added in this PR succeeds with PR #3179. I originally noticed the behavior described in #3225 when having a look at https://github.com/apache/pulsar/pull/24921. Now I tried to run the Pulsar tests with the snapshot version of #3179 but it skips a lot of tests (~900 of ~1,700 in "broker" group).