Ensure consistency in the test case execution environment

[mz1999]

I am running faces TCK tests in the GlassFish project using the following command:

mvn clean install -Ptck -pl :glassfish-external-tck-faces

Encountered failure while running the Issue2754IT test case.

[ERROR] Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.07 s <<< FAILURE! - in ee.jakarta.tck.faces.test.servlet30.ajax.Issue2754IT
[ERROR] ee.jakarta.tck.faces.test.servlet30.ajax.Issue2754IT.testAjaxViewScope  Time elapsed: 0.068 s  <<< FAILURE!
java.lang.AssertionError
	at ee.jakarta.tck.faces.test.servlet30.ajax.Issue2754IT.testAjaxViewScope(Issue2754IT.java:43)

View line 43 of test case Issue2754IT

assertTrue(page.asNormalizedText().contains("input: Validation Error: Value is required"));

This line of code is checking whether the return message of the page contains a specified string. Through debugging, it was found that in my runtime environment, the content returned by page.asNormalizedText() is:

Apparently, the return messages have been internationalized.

I went to check the mojarra project's definition of the return message, and in the English environment, the fields defined in the Messages.properties file are used :

jakarta.faces.component.UIInput.REQUIRED={0}: Validation Error: Value is required.

The environment I'm running tests in is macOS, and the country setting is "China", so I'll use the Messages_zh_CN.properties .

In fact, in this test case, the content of page.asNormalizedText() is as expected, but because it returns a Chinese string, it causes a wrong judgment.

We can keep GlassFish using consistent locale settings for any language environment on any operating system by setting the JVM parameters:

-Duser.language=en -Duser.country=US

This way all tests are run in a consistent environment.

Fortunately, GlassFish's arquillian plugin supports setting the JVM parameters via environment variables glassfish.systemProperties

private String systemProperties = System.getProperty("glassfish.systemProperties");

So, you can set the glassfish.systemProperties property in the pom:

<glassfish.systemProperties>
	user.language=en
	user.country=US
</glassfish.systemProperties>

This way, arquillian will convert glassfish.systemProperties to JVM parameters when starting GlassFish.

In addition, on macOS, the value of Locale.getDefault() cannot be influenced by setting the environment variable LC_ALL:

❯ LC_ALL=C jshell
|  ???? JShell -- ?? 17.0.6
|  ????????, ???: /help intro

jshell> Locale.getDefault()
$1 ==> zh_CN_#Hans

On Linux, it is possible to change the Locale.getDefault() via the environment variable LC_ALL.

$ jshell
|  欢迎使用 JShell -- 版本 17.0.6
|  要大致了解该版本, 请键入: /help intro

jshell> Locale.getDefault()
$1 ==> zh_CN

$ LC_ALL=C jshell
|  Welcome to JShell -- Version 17.0.6
|  For an introduction type: /help intro

jshell> Locale.getDefault()
$1 ==> en_US

So, I think setting JVM parameters is a more prudent approach to keep all environments consistent.

[mz1999]

Wow, I thought I am the only one who uses that ;-) And yeah, that's what I always have to do, change those env properties.

I did learn from you that it is possible to run tck tests in this way.