Investigate and find the best way to pass non-container layer tunables

[chandrams]

We need to investigate and find the best way to pass the non-container layer tunables.

For example, OPENJ9_JAVA_OPTIONS takes precedence over JAVA_OPTIONS - https://www.eclipse.org/openj9/docs/cmdline_specifying/

  {
            "name": "update env",
            "spec": {
               "template": {
                  "spec": {
                     "container": {
                        "env": {
                           "JVM_OPTIONS": "-XX:MaxInlineLevel=23",
                           "JVM_ARGS": "-XX:MaxInlineLevel=23"
                        }
                     }
                  }
               }
            }
       }

[kusumachalasani]

For JVM Tunables and Quarkus Tunables, right side parameters takes the precedence.

Order of precedence for JVM tunables :

• JAVA_TOOL_OPTIONS.
• OPENJ9_JAVA_OPTIONS (OpenJ9 specific)
• JDK_JAVA_OPTIONS
• Command line options ( -Xoptions: OpenJ9 specific)

Order of precedence for Quarkus tunables :

• System properties
• Environment variables
• File named .env placed in the current working directory
• application.properties file placed in the $PWD/config/ directory.
• An application configuration file, i.e. src/main/resources/application.properties

[dinogun]

@kusumachalasani It looks like we did not conclude on what is the best option to use from Autotune when we are constructing the env options. Can you please add your recommendation here and also update all benchmarks in the benchmarks repo to follow the same

[kusumachalasani]

Its decided to use "JDK_JAVA_OPTIONS" to pass the env options and is updated to use the same in benchmarks.