Allow JRE testing for other testsuite aside from JCK

[andrlos]

So basically what the problem is, is that we are unable to run JRE testing with anything but JCK, since JAVA and JAVAC command variables are only defined and propagated from the jck.mk... if we want to expand JRE testing on other testsuites we should find a suitable place to move this declarations to. This also targets the issue of headless testsuite https://github.com/adoptium/aqa-tests/issues/5224 , that was designed with jre testing in mind and is circumventing this in its own way. We should align this with the mechanics used elsewhere in Adoptium pipeline, in this case use the same mechanic for jre testing as used in JCK.

[andrlos]

Hi, so here is a thought: within aqa-tests we have the JenkinsfileBase groovy script that seems to be handling the basic test setup.. within this script there is a function called "setup_jck_interactives" I would split this function into setting the CUSTOMIZED_SDK_URL processing and setup of JAVA_TO_TEST that we would move from the jck/jck.mk to a new function inside the script and the other jck specific procedures that would be left inside the "setup_jck_interactives" function. This would allow execution of the new function outside of the jck context.

[andrlos]

for example setup_customized_sdk could be the name..

[andrlos]

or better yet.. what would be the harm in running the "setup_jck_interactives" function in non-jck test? just renaming a few things there.. there is suspiciously small amount of code that needs to be jck specific other than name-wise in my opinion

[smlambert]

We also support running from command line for those who do not use our Jenkins scripts, so the setup logic can not be specifically at that '2nd layer' of our 3 layer design.

The logic in jck.mk will be moved to a makefile that can then be included by all of the test group makefiles (jck.mk, openjdk.mk, system.mk, etc). Setting USE_JRE=true will be sufficient to utilize the supplied SDK for compiling tests, but the supplied JRE for executing tests.

[andrlos]

yea, but I thought that the goal was to not have to introduce a new mk file for functional testing... we would have to create a functional.mk (non-existent now AFAIK) to have a mk file to include it by.

[andrlos]

https://github.com/adoptium/aqa-tests/pull/5490 so is this how you would see it smlambert?

[smlambert]

I would probably not create a commonSetup.mk file in this repo, but rather leverage a make file in the TKG repository, likely https://github.com/adoptium/TKG/blob/master/testEnv.mk. In that way, it is available to 'vendor' test material held in repositories outside of the AQAvit repositories.

[andrlos]

oh okay.. will look into that

[andrlos]

How correct is my assumption that after inserting the code into testEnv.mk, I don't have to include it anywhere and the testEnv.mk would be already executed prior to every test run?

[smlambert]

Believe it is executed prior to every test run, but of course these changes will require 2 types of testing:

• validate whether this works via command line
• validate via a Grinder

[smlambert]

2 PRs needed, one in TKG and one in aqa-tests (changing testheadlessComponents command in its playlist to use JAVA_TO_TEST as the JRE/JDK to test).

Instructions for testing locally: https://github.com/eclipse-openj9/openj9/wiki/Reproducing-Test-Failures-Locally

[andrlos]

I first need to make sure that the jck is not affected when I change this..

[andrlos]

smlambert how about this? still need to test it tho.. it turns out the JAVA_COMMAND and JRE_COMMAND are not yet defined when testEnv.mk is called

https://github.com/adoptium/aqa-tests/pull/5492 https://github.com/adoptium/TKG/pull/596

[smlambert]

I will run some Grinders on our private Temurin-compliance Jenkins server to verify JCK tests can still be run, if you can do some local testing for the functional use case.