= TestNG Engine for the JUnit Platform :jcommander-version: 1.72 :jquery-version: 3.5.1 :junit-platform-version: 1.8.2 :surefire-version: 3.0.0-M7 :testng-min-version: 6.14.3 :testng-max-version: 7.10.2 :testng-engine-version: 1.0.5

Allows executing https://testng.org[TestNG] tests on the JUnit Platform

== Usage

.Console Launcher [%collapsible]

When running without a build tool, you need to download the following jars from Maven Central:

• https://search.maven.org/remotecontent?filepath=org/junit/platform/junit-platform-console-standalone/{junit-platform-version}/junit-platform-console-standalone-{junit-platform-version}.jar[junit-platform-console-standalone-{junit-platform-version}.jar]
• https://search.maven.org/remotecontent?filepath=org/junit/support/testng-engine/{testng-engine-version}/testng-engine-{testng-engine-version}.jar[testng-engine-{testng-engine-version}.jar]
• https://search.maven.org/remotecontent?filepath=org/testng/testng/{testng-max-version}/testng-{testng-max-version}.jar[testng-{testng-max-version}.jar]
• https://search.maven.org/remotecontent?filepath=com/beust/jcommander/{jcommander-version}/jcommander-{jcommander-version}.jar[jcommander-{jcommander-version}.jar]
• https://search.maven.org/remotecontent?filepath=org/webjars/jquery/{jquery-version}/jquery-{jquery-version}.jar[jquery-{jquery-version}.jar]

The following samples assume the above jars have been downloaded to the local lib folder and production and test classes to bin/main and bin/test, respectively.

[source,subs="attributes+"]

$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher
-cp bin/main -cp bin/test
--include-engine=testng --scan-classpath=bin/test

Thanks for using JUnit! Support its development at https://junit.org/sponsoring

╷ └─ TestNG ✔ └─ CalculatorTests ✔ ├─ add(int, int, int) ✔ │ ├─ [0] 0, 1, 1 ✔ │ ├─ [1] 1, 2, 3 ✔ │ ├─ [2] 49, 51, 100 ✔ │ └─ [3] 1, 100, 101 ✔ └─ addsTwoNumbers ✔ 2021-07-04T17:43:52.223145 description = 1 + 1 = 2

Test run finished after 38 ms [ 3 containers found ] [ 0 containers skipped ] [ 3 containers started ] [ 0 containers aborted ] [ 3 containers successful ] [ 0 containers failed ] [ 5 tests found ] [ 0 tests skipped ] [ 5 tests started ] [ 0 tests aborted ] [ 5 tests successful ] [ 0 tests failed ]

====

.Gradle [%collapsible]

[source,kotlin,subs="attributes+"] .build.gradle[.kts]

dependencies { testImplementation("org.testng:testng:{testng-max-version}") testRuntimeOnly("org.junit.support:testng-engine:{testng-engine-version}") // <.> } tasks.test { useJUnitPlatform() // <.> }

<.> Add the engine as an extra dependency for running tests <.> Configure the test task to use the JUnit Platform

.Maven [%collapsible]

[source,xml,subs="attributes+"] .pom.xml

=== Supported versions

==== TestNG

The engine supports TestNG version {testng-min-version} and above.

==== JUnit Platform

The engine requires at least JUnit Platform 1.5.x.

=== Configuration Parameters

The following JUnit Platform https://junit.org/junit5/docs/current/user-guide/#running-tests-config-params[configuration parameters] are supported.

==== Execution

testng.allowReturnValues (boolean):: whether methods annotated with @Test that have return values should be considered test methods (default: false; see https://testng.org/doc/documentation-main.html#test-methods[documentation]) + testng.dataProviderThreadCount (integer):: maximum number of threads to use for running data providers in parallel, if enabled via @DataProvider(parallel = true) (default: 10; see https://testng.org/doc/documentation-main.html#parameters-dataproviders[documentation]) + testng.excludedGroups (comma-separated list):: groups to exclude (see <<groups_vs_tags>>) + testng.groups (comma-separated list):: groups to be run (see <<groups_vs_tags>>) + testng.parallel (methods|tests|classes|instances|none):: TestNG's parallel execution mode for running tests in separate threads (default: "none"; see https://testng.org/doc/documentation-main.html#parallel-tests[documentation]) + testng.preserveOrder (boolean):: whether classes and methods should be run in a predictable order (default: true; see https://testng.org/doc/documentation-main.html#testng-xml[documentation]) + testng.threadCount (integer):: maximum number of threads for running tests in parallel, if enabled via testng.parallel (default: 5; see https://testng.org/doc/documentation-main.html#parallel-tests[documentation])

==== Reporting

testng.listeners (comma-separated list of fully-qualified class names):: custom listeners that should be registered when executing tests (default: ""; see https://testng.org/doc/documentation-main.html#testng-listeners[documentation]) + testng.outputDirectory (file path):: the output directory for reports (default: "test-output"; see https://testng.org/doc/documentation-main.html#running-testng[documentation]) + testng.useDefaultListeners (boolean):: whether TestNG's default report generating listeners should be used (default: false; see https://testng.org/doc/documentation-main.html#running-testng[documentation]) + testng.verbose (integer):: TestNG's level of verbosity for console output (default: 0)

=== Generating TestNG reports

.Console Launcher [%collapsible]

[source]

$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher
-cp bin/main -cp bin/test
--include-engine=testng --scan-classpath=bin/test
--config=testng.useDefaultListeners=true
--config=testng.outputDirectory=test-reports

====

.Gradle [%collapsible]

[source,kotlin,subs="attributes+"] .build.gradle[.kts]

tasks.test { useJUnitPlatform() systemProperty("testng.useDefaultListeners", "true")

val testNGReportsDir = layout.buildDirectory.dir("reports/testng")
outputs.dir(testNGReportsDir).withPropertyName("testng-reports")
jvmArgumentProviders += CommandLineArgumentProvider {
    listOf("-Dtestng.outputDirectory=${testNGReportsDir.get().asFile.absolutePath}")
}

}

====

.Maven [%collapsible]

[source,xml,subs="attributes+"]

=== Registering custom listeners

.Console Launcher [%collapsible]

[source]

$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher
-cp bin/main -cp bin/test
--include-engine=testng --scan-classpath=bin/test
--config=testng.listeners=com.acme.MyCustomListener1,com.acme.MyCustomListener2

====

.Gradle [%collapsible]

[source,kotlin,subs="attributes+"] .build.gradle[.kts]

tasks.test { useJUnitPlatform() systemProperty("testng.listeners", "com.acme.MyCustomListener1, com.acme.MyCustomListener2") }

====

.Maven [%collapsible]

[source,xml,subs="attributes+"]

== Limitations

[#groups_vs_tags] === Groups vs. Tags

https://testng.org/doc/documentation-main.html#test-groups[Groups] declared via the @Test annotation on test classes and methods are exposed as tags to the JUnit Platform. Hence, you can use https://junit.org/junit5/docs/current/user-guide/#running-tests-tag-expressions[tag filter expressions] to include or exclude certain groups. For example, given the following test class, including the tag included and excluding excluded will run test a and c but not b and d.

[source,java]

import org.testng.annotations.Test;

public class TestWithGroups { @Test(groups = "included") public void a() {} @Test(groups = {"included", "excluded"}) public void b() {} @Test(groups = "included") public void c() {} @Test public void d() {} }

However, since tags and therefore groups are filtered rather than selected, @BeforeGroups and @AfterGroups configuration methods are not executed on some versions of TestNG. Moreover, tag filters do not support wildcards such as integration.*. Therefore, instead of using tags, you can use the testng.groups and testng.excludedGroups configuration parameters to specify the groups that should be selected.

[WARNING]

Ideally, you should use either tags or the group configuration parameters. If you specify both, please ensure they match.

==== Using Tags

.Console Launcher [%collapsible]

[source]

$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher
-cp bin/main -cp bin/test
--include-engine=testng --scan-classpath=bin/test
--include-tag=included --exclude-tag=excluded

====

.Gradle [%collapsible]

[source,kotlin,subs="attributes+"] .build.gradle[.kts]

tasks.test { useJUnitPlatform { includeTags("included") excludeTags("excluded") } }

====

.Maven [%collapsible]

[source,xml,subs="attributes+"]

==== Using Configuration Parameters

.Console Launcher [%collapsible]

[source]

$ java -cp 'lib/*' org.junit.platform.console.ConsoleLauncher
-cp bin/main -cp bin/test
--include-engine=testng --scan-classpath=bin/test
--config=testng.groups=included --config=testng.excludedGroups=excluded

====

.Gradle [%collapsible]

[source,kotlin,subs="attributes+"] .build.gradle[.kts]

tasks.test { useJUnitPlatform() systemProperty("testng.groups", "included") systemProperty("testng.excludedGroups", "excluded") }

====

.Maven [%collapsible]

[source,xml,subs="attributes+"]

=== Suites

The engine's main intention is integration with build tools like Gradle and Maven. Hence, custom suites specified via testng.xml files are not supported.