Support some level of parallelization in junit-vintage-engine

[sarod]

Goal

Add support for some level of parallelization inside the Junit Vintage engine. Ideally the level of configuration should be similar to maven surefire parallel options.

Why

When including junit 5 tests in a project containing junit4 tests the natural path is to use junit 5 platform with the jupiter and vintage engines.

However this causes problem with parallelization. In a project with Junit4 tests only maven surefire level parallelization works fine. (See https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html). For Junit5 tests the jupiter engine parallel execution works great.

However when a project contains a mix of Junit4 and Junit 5 there is no great way to configure parallelism:

• Surefire parallel does not work with Junit Platform Provider so it's not an option
• Surefire forkCount or gradle maxParallelForks can be used but:
  • It's not available at all when using tycho surefire
  • It introduces some memory and execution time overhead compared to surefire parallel or junit 5 parallel.
  • It provides less control than Junit 5 parallel execution options.
• Combining fork + junit 5 parallel execution leads to more parallel executions than expected when executing junit 5 tests because parallelization is done at two level.

Another option is to configure two distinct test tasks one for junit 4 using surefire parallel and one for junit 5 but it kind of defeats the point of junit platform.

Deliverables

• [x] Not sure

[koretzki]

thanks, @sarod for rising this. I tried to fully embrace Junit 5 but had issues working it using the maven failsafe plugin 2.22.2-3.0.0-M4 (Junit 5.3-5.6.2), but I never could make parallelism to work even with settings that worked in maven surefire (in 3 ways to set them). The issue was the mixed Junit 4,5 testing environment. I moved them all to Junit 5 and it worked like magic in maven failsafe 2.2.22 with parallel execution.

[vab2048]

Are there any known workarounds to achieve this currently? I would like to run the junit4 classes in parallel and methods in the same thread.

[marcphilipp]

I would like to run the junit4 classes in parallel and methods in the same thread.

That should be fairly straightforward to implement. Basically, one would have to call RunnerExecutor concurrently and read a few configuration parameters to enable parallel execution and configure the number of threads: https://github.com/junit-team/junit5/blob/3f7fed61f2edcee2a5551dbde75382b1dbe21267/junit-vintage-engine/src/main/java/org/junit/vintage/engine/VintageTestEngine.java#L77-L84

[kishoretak]

@marcphilipp As you suggested running test classes in parallel with Vintage Engine is straightforward and the same is working fine. Is there any suggestion for running the test methods from the same class in parallel? I tried the solution mentioned here but it only supports test classes in parallel.

[nedtwigg]

I might be missing something, but so far as I can tell, it is not possible to run tests in parallel with junit-vintage. At the very least, junit.jupiter.execution.parallel.enabled=true does not seem to be effective. The PR #2449 looks small and high quality, I hope it or something similar gets merged someday.

If implementing the feature is very difficult, it might be nice if junit-vintage looked for junit.jupiter.execution.parallel.enabled=true and warned that it has no effect.

[nedtwigg]

For anybody else who wants this, I was able to get #2449 working on JitPack, took some tricks, you can use it like so:

maven { url 'https://jitpack.io' }

// testImplementation "org.junit.jupiter:junit-jupiter-api:${VER_junit5}"
// testImplementation "org.junit.vintage:junit-vintage-engine:${VER_junit5}"
testImplementation('com.github.diffplug.l0s-junit5:junit-jupiter-api:7786d92011e4c68255434b6f0f31cfd03841f24b')
testImplementation('com.github.diffplug.l0s-junit5:junit-vintage-engine:7786d92011e4c68255434b6f0f31cfd03841f24b')

test {
  useJUnitPlatform {}
  systemProperty 'junit.vintage.execution.parallel.enabled', 'true'
}

Swapping that little dep took a painpoint testsuite from 60 seconds to 10 seconds.