New extension with test templates

[svenkost]

I'm creating an extension for running tests to different instances.

The test class has an annotation that has all information to discover which instances should be tested. The annotation is a consul service name. I discover all the instances with the BeforeAllCallback and save them for later in the store that is in the ExtensionContext. (step 1)

The test class has a @TestTemplate test. With the TestTemplateInvocationContextProvider I generate a stream of TestTemplateInvocationContextObjects. In the object I have overridden the getDisplayName(final int invocationIndex). The stream is based on the list of environments I have created in step 1. (Let's call this step 2).

Then what happens is that the class will be instantiated with the TestInstanceFactory interface. The method createTestInstance(TestInstanceFactoryContext factoryContext, ExtensionContext extensionContext) is called; but here starts the problem: I have no indication for which test-(template-)instance the class is instantiated, so I have no reference for which environment this object is instantiated. (Step 3). I need to know why the object is to be instantiated, so I can call the constructor with the consul-information from step 1.

At first I made a 'dirty hack' to just have an internal counter to give every construction another environment; which works okay for 1 @TestTemplate, but if you have more than one and do it multithreaded, the environments are divided totally wrong: Test1-Env1 gets Env1, Test2-Env1 get Env2, Test2-Env2 gets Env3, Test1-Env2 gets Env4.

To run the test I use interceptTestTemplateMethod(Invocation<Void> invocation, ReflectiveInvocationContext<Method> invocationContext, ExtensionContext extensionContext) because I need to have initialization around the actual run of the test method.

What I need and can't find is that I need some sort of connection between the stream of tests that are created in step 2 and the construction of the test object for that specific test in step 3.

@ServiceName("myservice")
@ExtendWith(ConsulExtension.class)
@Execution(ExecutionMode.CONCURRENT)
class ExtensionTest  {
  private static final Logger logger = LoggerFactory.getLogger(ExtensionTest);

  private final ConsulTestContext consulTestContext;

  public ExtensionTest(final ConsulTestContext ctx) {
    logger.info("Constructor with ConsulTestContext: " + ctx);
    consulTestContext = ctx;
  }

  @TestTemplate
  public void myTestTemplate() {
    final long wait = Math.round(Math.random() * 3_000d) + 1_000L;
    logger.info("TestTemplate test " + consulTestContext.getMachine() + " class = " + this + ": waiting " + wait + " ms");
    try {
      Thread.sleep(wait);
    } catch (final InterruptedException ignored) {
    }
  }
}

Deliverables

• [ ] PR to provide the right ExtensionContext (the MethodExtensionContext instead of the ClassExtensionContext) to the constructor if it is instantiated by a @Test or @TestTemplate - https://github.com/junit-team/junit5/pull/2961

[Michael1993]

Hey @svenkost, I think I finally figured out what you want to do!

If I understood correctly, you want to supply your test with values you store in your TestTemplateInvocationContext. The simplest way you can do that is by overriding the getAdditionalExtensions method in your TestTemplateInvocationContext and returning an anonymous ParameterResolver instance there. Here would be my version of your example:

@ServiceName("myservice")
@ExtendWith(ConsulExtension.class)
@Execution(ExecutionMode.CONCURRENT)
class ExtensionTest  {
  private static final Logger logger = LoggerFactory.getLogger(ExtensionTest);

  @TestTemplate
  public void myTestTemplate(Machine machine) {
    final long wait = Math.round(Math.random() * 3_000d) + 1_000L;
    logger.info("TestTemplate test " + machine + " class = " + this + ": waiting " + wait + " ms");
    try {
      Thread.sleep(wait);
    } catch (final InterruptedException ignored) {
    }
  }
}

Then in your ConsultTestTemplateInvocationContext:

public class ConsulTestTemplateInvocationContext {
  private final Machine machine;
  // constructor, other methods, etc.

  @Override
  @Override
  public List<Extension> getAdditionalExtensions() {
    return Arrays.asList(new ParameterResolver() {
      @Override
      public boolean supportsParameter(ParameterContext parameterContext, ExtensionContext extensionContext) throws ParameterResolutionException {
        return parameterContext.getParameter().getType().equals(Machine.class);
      }

      @Override
      public Object resolveParameter(ParameterContext parameterContext, ExtensionContext extensionContext) throws ParameterResolutionException {
        return ConsultTestTemplateInvocationContext.this.machine;
      }
    });
  }
}

Does this help with your problem?