apache
[WIP] Use junit platform runner to run junit 3, junit 4 and TestNG tests via vintage engine and testng engine
Overview / Goal
- The PR intends to switch Surefire to use the JUnit Platform runner (i.e. the JUnit 5 infrastructure) to run JUnit 4 tests via the Vintage engine, and also support TestNG under that runner.
- The main changes are removal support for dependency to JUnit 3, for versions of JUnit 4 older than 4.12 and testng older than 6.14.3
- main advantage will be to have only a single implementation to maintain as listener/reporter/launcher of test running platform. Without such change there 5 implementations to maintain which makes any changes extra work which doesn't help to get new contributors to help.
Major Changes
| Change Description | Impact |
|---|---|
| Drop JUnit 3 provider | The PR removes the “JUnit 3 provider” entirely. Junit3 code will be executed via junit vintage only need to upgrade junit dependency to 4.12 at least |
| Enforce minimum JUnit 4 version | JUnit 4.12 becomes the minimum supported version. Versions earlier than 4.12 are no longer supported. |
| Route JUnit 4 tests via JUnit Platform + Vintage engine | Instead of using a separate JUnit 4 runner / provider, JUnit 4 tests will be executed via the JUnit Platform infrastructure using the Vintage engine. |
| Integrate TestNG under JUnit Platform runner | TestNG will now also be run via JUnit Platform, with mappings of TestNG configuration (groups, etc.) to the JUnit Platform infrastructure.\ |
| Code removal / cleanup | A lot of internal code for the old JUnit 3/4/47/TestNG providers is removed for simplification |
| Version bump / snapshot change | The version is bumped to something like 3.6.0-SNAPSHOT in the PR. |
| Minimum TestNG version | The PR also sets a minimum supported TestNG version (6.14.3) |
| Adaptation in existing features | Some features are reworked or adapted (e.g. parallel execution, mapping of “groups” in TestNG to tags/filters, error handling in logs) so they work properly under the new runner approach. |
There may be compatibility issues or subtle behavior changes when running JUnit 4 or TestNG tests under the new infrastructure (e.g. differences in how filters, groups, or lifecycle behaviors are handled).
The change consolidates execution under the JUnit Platform, which could simplify the runner architecture
@olamy I did not see any positiv contribution from your site, only removals in favor of JUnit5. The JUnit5 had one tendency 6 years ago, it was removal of Surefire and acceptance of "their" plugin. I know it because we talked about it that time. If you follow this approach, you would break users, you would break the default behavior. If you think tat Surefire Provider JUnit4 and Surefire Provider JUnit4.7+ are only about the version, then you do NOT understand this project. If you think that Surefire Provider JUnit3 is only about JUnit3 then you again do not understand the point. I saw people who remove something because they think that the old must go away but it happens because they are maybe corrupted or they do not want to understand, they do not listen and nothing exists for them only their ambition. So, better for you is to collaborate with people who have long experiences with this project, who have the continuity of history and not to waste the energy with commits because they are really useless. On the sire of Surefire there was a plan and the plan had a purpose but somebody did not understand it and deleted it. Without this plan some bugs cannot be fixed. Yesterday I talked to one contributor, he made a PR few days ago, and he understood point within several minutes but it is very hard to understand for someone who want's to do arrogant commits ignoring what I am telling you some weeks.
The JUnit5 had one tendency 6 years ago, it was removal of Surefire and acceptance of "their" plugin. I know it because we talked about it that time.
I assume you mean the "JUnit5 team"; the people behind the project. Right?
If yes, we had "that time" and still have today more than "one tendency". We provided and still provide help, options, and alternatives for many tool authors and vendors to adopt to the (then new) JUnit Platform. Nothing changed there, we still do it happily and will do so in the foreseeable future.
Are you mixing up the junit-platform-surefire-provider provided by the team (to support Surefire!) and my junit-platform-maven-plugin which was attempt to create Maven plugin talking directly to the JUnit Platform and playing nice with Java modules?
I saw people who remove something because they think that the old must go away but it happens because they are maybe corrupted or they do not want to understand, they do not listen and nothing exists for them only their ambition.
"corrupted"? The prefixed "maybe" doesn't make it sound any better.
While all participants should adhere to this code of conduct, we recognize that sometimes people may have a bad day, or be unaware of some of the code's guidelines. When that happens, you may reply to them and point out this code of conduct.
https://www.apache.org/foundation/policies/conduct
@Tibor17 : it has been reminded a few times that peoples who need JUnit 3 can stay on old versions of Surefire plugin. I have not seen a counter-argument to that. I have not seen neither a counter-argument to the fact that supporting JUnit 3 is an impediment to the development of new features. To recap:
- Why users who want to stay on old JUnit versions could not stay also on old Surefire versions?
- Considering that removing JUnit 3 support is necessary for making improvements easier, why supporting JUnit 3 in latest Surefire is more important than allowing the development of new features (multi-release support, etc.)?
Please note Junit3 code can be still executed but need the junit 4.12 dependency at least
Just to point out: according to the JUnit 6 release notes, the JUnit Vintage engine is deprecated and (to quote) "it should only be used temporarily while migrating tests to JUnit Jupiter or another testing framework with native JUnit Platform support".
What would the ramifications be for Surefire users that still use JUnit 3/4 tests if this change goes through & then later the JUnit folks decide to abruptly drop the JUnit Vintage engine in a future JUnit release?
Just to point out: according to the JUnit 6 release notes, the JUnit Vintage engine is deprecated and (to quote) "it should only be used temporarily while migrating tests to JUnit Jupiter or another testing framework with native JUnit Platform support".
What would the ramifications be for Surefire users that still use JUnit 3/4 tests if this change goes through & then later the JUnit folks decide to abruptly drop the JUnit Vintage engine in a future JUnit release?
You will have a few ways to go about this:
-
Option 1: Easiest one, just stick with the last Surefire version before this PR is merged by setting it in your pluginManagement. Since you’re already on the older JUnit 3/4 (which are deprecated anyway), it’s totally fine to stay on an older Surefire for now.
-
Option 2: Surefire will pick up whatever JUnit version or platform it finds (JUnit, JUnit Platform, or JUnit Vintage) from your pom. So unless you’re moving to JUnit 6.x (or a Vintage version that dropped JUnit 3/4 support), you should be all good.
-
Option 3: If you’re thinking about updating things, you could use a tool like OpenRewrite or even an AI helper to migrate your tests to JUnit 5 or 6.
What would the ramifications be for Surefire users that still use JUnit 3/4 tests if this change goes through & then later the JUnit folks decide to abruptly drop the JUnit Vintage engine in a future JUnit release?
It's deprecated, but not for removal. Not slowly, not abruptly, not "everly".
We think of it more as we do of JUnit 3 and 4: they are here to stay. Until the last user vanishes. So does the glue-code in junit-vintage-engine. As long as the JUnit Platform is supported by tools (including Maven), you get all the integration with your testing framework of choice. Through an implementation of a test engine. And in case of JUnit 3 and 4, both projects are in an end-of-life status, no change is to be expected on their side.
Not discussed with the team, yet, but I could think of moving the junit-vintage-engine module as sibling right next to the testng-engine project, making it a standalone top-level project.