Generic logic to get get and store benchmark metric

[llxia]

trafficstars

Extend based on https://github.com/adoptium/aqa-tests/pull/6339/, make checkDacapoH2Metric() and archiveDacapoH2Metric() generic.

On second thought, it may be better to have a separate script (i.e., Python - benchmarkMetric,py) to handle this.

• In benchmarkMetric.py, it contains a generic function checkMetric(). checkMetric() takes job console output and reads benchmarkMetric.json and runs each regex, and stores into one JSON output.

benchmarkMetric.json example:

[
    {
        "Benchmark": "dacapo-eclipse",
        "metrics": 
        {
          "dacapo-eclipse" : [
            {"regex": "DaCapo.*eclipse\\sPASSED\\sin\\s(\\d*\\.?\\d*)\\smsec"},
            {"higherbetter": true},
            {"units": "msec"}
        ]
      }
    },
{
        "Benchmark": "dacapo-h2",
        "metrics": 
        {
          "dacapo-h2" : [
            {"regex": "DaCapo.*h2\\sPASSED\\sin\\s(\\d*\\.?\\d*)\\smsec"},
            {"higherbetter": true},
            {"units": "msec"}
        ]
      }
    }
]

Note: Backslashes must be double-escaped (\\) in JSON strings.

Output: JSON file as specified in https://github.com/adoptium/aqa-tests/issues/6327. For example:

[
    {
        "Benchmark": "dacapo-eclipse",
        "metrics": 
        {
          "dacapo-eclipse" : [ 
            {"value": "123456"},
            {"higherbetter": true},
            {"units": "msec"}
        ]
      }
    },
{
        "Benchmark": "dacapo-h2",
        "metrics": 
        {
          "dacapo-h2" : [
            {"value": "4689"},
            {"higherbetter": true},
            {"units": "msec"}
        ]
      }
    }
]

related: https://github.com/adoptium/aqa-tests/issues/6327

top level issue: https://github.com/adoptium/aqa-tests/issues/6268

@smlambert @LongyuZhang if I missed anything, please feel free to update.

[llxia]

For getting console output, try myjob.getRawBuild().getLog()

https://stackoverflow.com/questions/31521030/output-the-console-text-of-a-jenkins-job-that-my-job-is-running https://stackoverflow.com/questions/42510489/console-output-in-pipelinejenkins

[MattyWeee123]

Hi, may I work on this?

[llxia]

In RHEL9 (LABEL: ci.role.test&&hw.arch.x86&&sw.os.linux&&sw.os.rhel.9), python is installed.

00:00:03.191  + python --version
00:00:03.191  Python 3.9.21

Internal job

There is no RHEL9 at Adoptium. @smlambert do you happen to know which machine has python installed by default?

[smlambert]

There is no RHEL9 at Adoptium. @smlambert do you happen to know which machine has python installed by default?

Possibly this one (which I think is recently added and still needs to have an updated label to suit our labeling schema)? test-rhibmcloud-rhel9-x64-1

[MattyWeee123]

@smlambert could I have permission to use currentBuild.rawBuild.getLog()

[smlambert]

In script approval is done now for getLog() if you want to try again @MattyWeee123 !