junit4

junit4 copied to clipboard

I want to be able to create my own runners for parameterized tests. This was possible, when PR #564 was merged. This was reworked in #773, so that user can use @RunWiht(Parameterized.class) followed by @UseParametersRunnerFactory. This is fine for simple scenarios. Our scenario is as follows: We have class extending Suite (let's call it SuperSuite), which reads some configuration and creates runners for each class in that suite, if it satisfies that configuration(we don't want some classes to be run with some configurations and vice versa). The tree of runners looks like this:

SuperSuite
 |___Config1Runner
 |        |____Class1
 |        |          |____method1
 |        |          |____method2
 |        |____Class2
 |                  |____method1
 |                  |____method2
 |___Config2Runner
         |____Class1
         |          |____method1
         |          |____method2
         |____Class2 (skipped, did not meet configuration requirement)

Now I want to implement mechanism of parameterized classes. Let's say I want Class 1 to be parameterized. The tree would look something like this:

SuperSuite
 |___Config1Runner
 |        |____Class1 -parameterized
 |        |          |____Class1 -param1
 |        |          |            |____method1
 |        |          |            |____method2
 |        |          |____Class1 -param2
 |        |          |            |____method1
 |        |          |            |____method2
 |        |          |____Class1 -param3
 |        |                       |____method1
 |        |                       |____method2
 |        |____Class2
 |                  |____method1
 |                  |____method2
 |___Config2Runner
         |____Class1 -parameterized
         |          |____Class1 -param1
         |          |            |____method1
         |          |            |____method2
         |          |____Class1 -param2
         |          |            |____method1
         |          |            |____method2
         |          |____Class1 -param3
         |                       |____method1
         |                       |____method2
         |____Class2 (skipped, did not meet configuration requirement)

I hope I described my intentions clearly.

With current situation I was almost able to do it: I added @UseParametersRunnerFactory(MyFactory.class) annotation to Class1 and in the SuperSuite I've created the Parameterized programatically for classes annotated with @UseParametersRunnerFactory. Unfortunately I need to pass some parameters to my MyFactory to be able to correctly create child runners (This is the part I'm unable to do right now).

Now for solutions: With PR #564, the solution would be pretty simple. I would just create my own Parameterized suite (extending Parameterized) overriding createRunner method.

Or... Put back constructor protected Parameterized(Class<?> klass, ParametersRunnerFactory runnerFactory) from PR #773.

I hope I made myself clear and sorry for the long post.

On second thought, putting back the protected constructor wouldn't help, because I have to pass to that factory some parameters (specific for each configuration to be able to create the runners properly), but Parameterized is always using non parametric constructor.

What kind of parameters do you need to pass to MyFactory? Do they depend on the configuration?

Yes they do. So far I've created a workaround in my solution: The SuperSuite is unchanged, under it there is MyParameterized extending Parameterized and under it are MyParameterizedRunners extending BlockJunit4Class. MyParameterizedRunners have fields and setters for them. Then I overrided MyParametrized's method getChildren:

@Override
protected List<Runner> getChildren() {
    List<Runner> children = super.getChildren();
    for (Runner runner : children) {
        if (runner instanceof MyParameterizedRunner) {
            MyParameterizedRunner testRunner = (MyParameterizedRunner) runner;
                         testRunner.setFiled1 = <value>;
                         testRunner.setField2 = <value>;
                         ...
        } else {
            return null;
        }
    }
    return children;
}

This basically allows me to put required fields into MyParameterizedRunner, but it would be much better if this could be somehow possible to achieve via factory class & MyParameterizedRunner's constructor.

Do only Parameterized classes need config parameters or regular tests, too?

Yes they do. The "regular" tests in our suite are not using simple BlockJUnit4Class runner, but our own runner (extending BlockJUnit4Class).

Can you provide some insight on what the configuration is actually for? I'm having a hard time thinking of possible examples.

We're writing UI tests for eclipse (JBoss Tools/JBoss Developer Studio).

We have mechanism called Requirements, which is basically an annotation on class and some code. This Requirement is something I want to do on lots of testclasses. One of the simplest requirements that we have is @OpenPerspective(JavaPerspective.class). This just opens Java perspective in prior of executing this test class.

More complicated requirement could be @ApacheTomcarServer(state=SerReqState.RUNNING). This requirement adds apache tomcat server to eclipse and starts it in prior of test class execution.

This is where configurations come to play: Configuration is XML file. For ApacheTomcatServer requirement to work properly, one has to define in configuration what is that server version and what is it's location.

So an example of test, which deploys some project to different tomcat servers could be as follows:

package test.pkg;

<imports>

@RunWith(RedDeerSuite.class)
@ApacheTomcatServer(state=ServerReqState.RUNNING)
public class MyTest {

        @InjectRequirement
        public ServerRequirement req;

        @Test
        public void testDeployProject(){
                Project project = createProject();
               deployProjectToServer(project, req.getServerNameLabelText(req.getConfig()));
                checkDeployedProject();
        }
}

When I start this tests with system property pointing to folder with multiple config XMLs (one for each server), the execution tree would look like:

RedDeerSuite
 |___ApacheTomcat6
 |       |___MyTest *
 |                |_testDeployProject
 |___ApacheTomcat7
 |       |___MyTest *
 |                |_testDeployProject
 |___ApacheTomcat8
         |___MyTest *
                  |_testDeployProject

In this example, Runner which is in the tree above marked with "*" has to have some mechanism to execute code. This code is not same every time. It differs with different Requirements. In our current implementation, were passing those Requirements to that runner (otherwise it's just extension of BlockJUnit4ClassRunner, with added evaluation of those Requirement in withBeforeClasses.

This approach is not working when I tried to rewrite our framework to be able to run with parameterized tests. There is no easy way how I could pass something like Requirement to that Runner.

I've managed to create some workaround in our code (PullRequest - https://github.com/jboss-reddeer/reddeer/pull/523/files). The runner matching the runner marked as "*" is ParameterizedRequirementsRunner. It's parent runner is ParameterizedRunner. The workaround is that I'm not passing those parameters I need to pass to it's child runner by constructor (because it's created by factory and it has to have one-parameter constructor), instead I'm passing those parameters in ParameterizedRunner.getChildren() method. This is the only place I could find, where I can somehow modify the ParameterizedRequirementsRunner.

I don't like this solution, but I can't think of anything else right now.