aqa-tests icon indicating copy to clipboard operation
aqa-tests copied to clipboard

Generalized regression monitoring for PerfPipeline.

Open MattyWeee123 opened this issue 5 months ago • 4 comments

metricConfig2JSON.py parses and rearranges BenchmarkMetrics.js into JSON. initBenchmarkMetrics.py generates templates for a groovy aggregate metric map and a single test map for archiving individual test/baseline child jobs. benchmarkMetrics.py updates a single test map with the metric value for a run.

testList is used to generalize regression monitoring and EXIT_EARLY for child jobs that run multiple tests (removing from testList will have both parent and child ignore the removed test).

Using specific rhel9 machine to run the python scripts and getPythonDependencies uses curl to retrieve the relevant python scripts.

BenchmarkMetric.js needs to be updated (lusearch-fix should be lusearch, tomcat is not there so the script thinks it should grab all dacapo metrics for tomcat which is not true, ...)

Fixes: #6418, #6355

MattyWeee123 avatar Jul 21 '25 18:07 MattyWeee123

Thanks @MattyWeee123 . Have you tried to run this in https://ci.adoptium.net/view/Test_grinder/job/Perf_Pipeline/? If so, please provide the link. Thanks

llxia avatar Jul 28 '25 16:07 llxia

@MattyWeee123 Will need a rebase (or that merge conflict fixed)

karianna avatar Jul 30 '25 23:07 karianna

https://ci.adoptium.net/view/Test_grinder/job/Perf_Pipeline/178/ to check that resolving merge conflicts did not alter behaviour.

smlambert avatar Dec 03 '25 14:12 smlambert