junit5
junit5 copied to clipboard
junit-team
Introduce `@BeforeSuite` and `@AfterSuite` annotations
I suggest that @BeforeSuite and @AfterSuite annotations should be added to JUnit 5.
Obviously they should execute before and after the whole suite.
Or is there some other mechanism to accomplish that?
I suggest that there should not be static restriction, i.e. also non-static methods could be annotated with @BeforeSuite and @AfterSuite.
I personally prefer non-static as we use Spring in our tests and want to use injected beans in @BeforeSuite method.
Related to #163.
There's not always an easy way to control what's considered a "Suite". If you look at the Maven plugin, it controls the inclusion/exclusion of tests via XML. The JUnitPlatform test runner allows a suite to be defined declaratively but it's not honored by the Maven plugin.
If there was always a declarative way to run a suite, I'd say that @BeforeAll/@AfterAll could just be reused on the suite declaration. As things stand and in the name of unambiguous behavior, I prefer your annotations.
In my consideration "Suite" can be defined via:
- Maven/Gradle plugin inclusion/exclusion
-
JUnitPlatformdeclarative declaration - IDE when I click Run on the whole package of tests
In all those cases I would expect that @BeforeSuite/@AfterSuite would be executed just once before/after the whole bunch of tests, for example to perform time consuming operation such as database cleanup.
@ondrejlerch How would JUnit find @BeforeSuite/@AfterSuite annotated methods? For example, when you run a test class from an IDE would it only look at the test class or someplace else?
The Store abstraction along with the root ExtensionContext provides a means to prepare something once and share it between multiple test classes. However, there's no way to clean it up at the moment (that depends on #742).
@marcphilipp you could consider a solution such as Jandex, which provides you an index over all annotations
Well, in any case, it has to be obvious for test authors where JUnit will actually look for these annotations, i.e. if they will be applied when they execute a test class.
environment (shell/build tool/IDE/...) (1)
platform/launcher (DefaultLauncher) (2)
engine a (Jupiter) (3a)
...
engine x (3x)
engine z (3z)
I see an easy way to implement an extension point for 3a -- which frames the all tests executed by the Jupiter engine with a "before the first" and "after the last" callback.
Pseudo-code for JupiterTestEngine:
@Override
public final void execute(ExecutionRequest request) {
// TODO context = createExecutionContext(request);
// try {
// TODO callBeforeExecutionCallbacks(context);
new HierarchicalTestExecutor<>(request, context).execute();
// } finally {
// TODO callAfterExecutionCallbacks(context);
// }
}
For the record, I actually would not use the annotation specifically. I am more interested in the corresponding Extension (especially AfterSuiteCallback).
I was just answering the general question of "finding annotations". That is what Jandex excels at, whether you want to use it or not...
I’m fine with adding an extension point that uses the existing registration mechanisms.
That should work for everything except Maven Surefire... see #1113.
As I would never use Maven ever again, I can live with that ;)
But yes, that is exactly what would work best for us:
@ExtendWith( MyAfterSuiteCallback.class )
...
Started working on a PR that introduces BeforeExecutionCallback and AfterExecutionCallback...
@marcphilipp
To your question: How would JUnit find @BeforeSuite/@AfterSuite annotated methods? For example, when you run a test class from an IDE would it only look at the test class or someplace else?
I suggest answer: JUnit should search among all executed test classes, not someplace else. So if you execute just one test class (via IDE or Maven or Gradle) it should look only there. However, if you execute multiple test classes it should look at all of them and and run @BeforeSuite first.
If we use JUnit 5 with Spring then Spring Context should be initialized first, then @BeforeSuite should be executed, so that we can use Spring in @BeforeSuite. Obviously @AfterSuite should be executed before Spring Context is closed for the same reason.
BeforeExecutionCallback and AfterExecutionCallback sound also good.
Solution should work with IDE, Maven and Gradle as those are most typical tools for test execution. I agree with @sebersole that Maven is not cutting edge, however, if Maven Surefire would not work then please suggest what would work in Maven as it is widely used.
Thanks!
I'm also fine with the idea of introducing new BeforeExecutionCallback and AfterExecutionCallback extension APIs.
But... we'll have to work on the names. 😉
That and the implementation and semantics of course.
Hi everybody,
I'd like to point out that we have to be very cautious here about the use of the term "suite" for a feature like this.
#744 is the intended solution for suites on the Platform.
Whereas, this issue is about before and after callbacks that are specific to a given run of the Jupiter TestEngine.
So please keep this in mind so that we don't hinder the applicability and utility of whatever is eventually done for #744.
Thanks,
Sam
Given that clarification, maybe BeforeTestEngineExecutionCallback and AfterTestEngineExecutionCallback?
This StackOverflow question provides an interesting use case (although there may be better solutions than @BeforeSuite):
We are using Maven to run JUnit5 functional tests. Most of the tests require data to go through our entire system (a Storm topology) which could take around 1 minute. We need to send some data through and then wait 1 minute before we run the tests.
We need to stage the data and then communicate keys and such to the tests.
Is there a way to run a method/class to stage the data, then wait one minute, then run the rest of the tests?
I have a case quite similar to the one named by @nicolaiparlog. I work very close to some SQL and NoSQL databases and testing these entities are usually quite hard. If you are using MySQL/Postgres/Mongo you have three options:
- Running tests agains a mocked or embedded database. This can be acceptable, but you are not testing exactly what you want
- All developers and your CI have access to required databases (maybe local installations or a shared remote one). In this case you must be very careful. Your tests must be prepared to be running concurrently (good look testing DDL sentences with MySQL) and they must leave, in any situation, the database on a correct state.
- You start a new database instance each time you launch a test. You can do it with some libraries that automatically unzip and starts the database for you or you can start a Docker container with exactly what you need (including initial data, which is fantastic). You can start the database once before you run your suite or once per test method/class. The latter can be done using JUnit annotations (there are some alternatives, I have my own that starts a docker container using Junit extensions, but there are some others. The first can be done manually or with Gradle/Maven, but I would like to be able to create a JUnit extension that launches the container when the suite starts and removes it when it finishes.
Running those use-cases in single class might be an option. Then you may make use of @BeforeAll and friends. If the single class grows too large, well, you may split it up and care about starting your "setup" only once.
I'm running these cases on a single class right now, but once you want to split the class, before/after suite could really simplify the code (and I can distribute a JUnit plugin so my coworkers can easily reuse my code on their projects).
BTW: A modern machine can start an empty MySQL/Postgres/Mongo container in a couple of seconds, so it is not really bad to start one per class. But the faster it goes, the more frequently you can run our mini-integration tests.
Keep in mind that an extension that implements BeforeAllCallback and is registered globally via the ServiceLoader mechanism can make use of CloseableResource and the ExtentionContext.Store to achieve many suite-like behaviors.
See #1555 and the discussion on Gitter linked from there for further details.
hi All!
Instead of @BeforeSute I used like this:
Add to your Base abstract class (I mean abstract class where you initialize your driver in setUpDriver() method) this part of code:
private static boolean started = false;
static{
if (!started) {
started = true;
try {
setUpDriver(); //method where you initialize your driver
} catch (MalformedURLException e) {
}
}
}
And now, if your test classes will extends from Base abstract class -> setUpDriver() method will be executed before first @Test only ONE time per project run.
@SergiiNik Nice suggestion! However AFAICT, it only works when running tests one after another (which is the default behaviour) instead of in parallel.
By my understanding, when running in parallel, the code setUpDriver() may get called multiple times before started = true becomes visible to all threads running the tests, so the double-checked locking pattern with a Lock may be needed in this case. :)
...although, I think the holder idiom only works if setUpDriver() returns something... 🤔
@SergiiNik Nice suggestion! However AFAICT, it only works when running tests one after another (which is the default behaviour) instead of in parallel.
By my understanding, when running in parallel, the code
setUpDriver()may get called multiple times beforestarted = truebecomes visible to all threads running the tests, so the double-checked locking pattern with aLockmay be needed in this case. :)
Hello @jbduncan , we have our own almost similar with @SergiiNik solution and it works perfect in parallel, we use the following Junit parameters:
junit.jupiter.execution.parallel.enabled=true
junit.jupiter.execution.parallel.mode.default=concurrent
junit.jupiter.testinstance.lifecycle.default=per_class
See my gist:
https://gist.github.com/Legionivo/23313f5998284e25af9a4eb6570a6360
Hi guys! Do we have a chance to get beforeSuite/afterSuite feature done in the near future? Or for some reason, it is postponed?
