TestNG read surefire systemPropertyVariables configuration from first execution if global is absent

[michaldo]

Plugin Version

TestNG 6.10.0.201612030230 org.testng.eclipse.feature.group Cedric Beust

(the same behavior for TestNG 6.10.1.201701311310 org.testng.eclipse.feature.group Cedric Beust)

<build>
	<pluginManagement>
		<plugins>
			<plugin>
				<artifactId>maven-surefire-plugin</artifactId>
				<version>2.19.1</version>
				<executions>
					<execution>
						<id>private-execution</id>
						<configuration>
							<systemPropertyVariables>
								<abc>private</abc>
							</systemPropertyVariables>
						</configuration>
					</execution>
				</executions>
			</plugin>
		</plugins>
	</pluginManagement>
</build>

TestNG/Maven configuration: pass argline, systemPropertyVariables and environmentVariables checked

Expected behaviour

When I run a test from Eclipse with code System.out.println("abc:" + System.getProperty("abc")); I expect

abc:null

because global configuration is empty and only specific configuration private-execution has abc configured

In my real project I have several executions for particular cases but I don't want first execution configuration be activated for Eclipse

Actual behaviour

abc:private

The Dependency Management tool for your project

• [x] Maven
• [ ] Gradle
• [ ] Ant
• [ ] Eclipse Buildpath (aka. Use "TestNG Library" for your project in Eclipse)

Operating System

• [x] Windows
• [ ] Linux
• [ ] OSX

[missedone]

@michaldo, could you share more detail about your expectation on your use case? do you want to use execution level configuration (if any) for the right test? or could you share a more complete example of the maven-surefire-plugin configuration, this will help to figure out a solution to rule all scenarios (hopefully). for now it's hard to define a complete rule to satisfy various scenario, as you can see the rule today is simple and dirty that the first occurred configuration win (even it's not the right one for the specific test).

[michaldo]

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
<!--begin global section -->
    <configuration>
    <systemPropertyVariables>
        <drink>water</drink>
    </systemPropertyVariables>
    </configuration>
<!--end global section -->
    <executions>
        <execution>
            <id>party</id>
            <configuration>
                <systemPropertyVariables>
                    <drink>beer</drink>
                </systemPropertyVariables>
            </configuration>
        </execution>
        <execution>
            <id>relax</id>
            <configuration>
                <systemPropertyVariables>
                    <drink>juice</drink>
                </systemPropertyVariables>
            </configuration>
        </execution>
    </executions>
</plugin>

With configuration above I expect that when I launch testng from Eclipse then drink=water. That is actutal behavior, that is OK. Let say I modified my code following convention-over-configuration principle and now default drink is water. I can remove global-section. When I launch testng from Eclipse then drink=beer. Isbeer better than juice? Conclusion: When I run TestNG from eclipse, I cannot define that I want use configuration from particular execution. Instead, I use global configuration. When global configuration is empty, I expect TestNG will run with empty global configuration, not with configuration of first possible execution

[missedone]

When global configuration is empty, I expect TestNG will run with empty global configuration, not with configuration of first possible execution

do you mean you don't want to append any systemPropertyVariables to the runtime testng process when global configuration is empty while even execution configuration is available? if so, could you disable it by right click on the project -> Properties -> TestNG -> Maven -> check "Enable project specific settings" -> uncheck "systemPropertyVariables" ?

[michaldo]

That is wrong way to solve the case. Assume case that I have one execution extremely tuned for performance with disabled logging; log4j.configuration=off-log4j.properties. Assume that I have also system property which decide about db driver used for test. One day I decided to support only one driver, so I remove systemPropertyVariables from global configuration (there were only one entry). Suddenly, all developer lost TestNG logging (because extremely tuned execution sneak into Eclipse setup) and I have broadcast message that they have to modify the project, each time they will import it to Eclipse. And next week I decide to add other system property variable to test. I have broadcast message that everybody must revert the modification. Soon nobody will know which project settings is valid

[missedone]

OK, as a workaround, could you add a placeholder system property in global configuration? for example:

<plugin>
    <artifactId>maven-surefire-plugin</artifactId>
<!--begin global section -->
    <configuration>
    <systemPropertyVariables>
        <testng_placeholder></testng_placeholder>
    </systemPropertyVariables>
    </configuration>
...

[missedone]

@michaldo, hope the workaround works for you.

At the meanwhile, I'm thinking a fix for this. The idea is to introduce a testng eclipse property that you can define in the maven pom, to be able to define the strategy of how to locate the configuration to append to runtime testng process. For example,

<properties>
<testng.eclipse.conf.locator>execution: execution_id</testng.eclipse.conf.locator>
</properties>

The locator value can be:

• none
• execution:{execution_id}
• ... To be determined

The advantage is, it's flexible enough on the value, and can make use of maven profile that the active one overrides the default. But the disadvantage is that, I really don't like to leak IDE specific properties to maven pom. What would you say? Or any other suggestions? Thx

[michaldo]

1. About workaround: it works.
2. About complete solution: Build has 2 users, developers and CI, generally human and machine. Human loves rely on defaults, machine has no problem with specify each execution hunders of properties. Therefore I think default surefire settings are for human and execution specific settings are for machine. Conclusion: Eclipse settings are default settings

[fragonib]

Hello,

When both surefire and failsafe plugins deliver for instance <systemPropertyVariables> configuration, which one is selected by TestNG M2E integraton plugin?

Thank you.

[michaldo]

It depends on plugin order. For example, I have master pom with build->plugins->surefire to be sure that surefire is always on first place.

[fragonib]

I understand plugin document order declaration matters.

Thank you for your quick answer.