aqa-test-tools
aqa-test-tools copied to clipboard
Update Benchmark Versions & Add Profiling Parameters to PerfNext Launcher
Problem Description
Currently, all benchmark configs are using older version of benchmarks. We should update them to these latest benchmarks.
| Current Version | Version to Use | |
|---|---|---|
| Liberty | 19.0.0.2 | 19.0.0.9 |
| ODM | 8.8.1 | 8.10.0 |
| SPECjbb2015 | SPECjbb2015GMR | specjbb2015v101_jaxb24 |
Also, we recently extended the support for profiling tools to ODM from Liberty. In future, this profiling ability would be extended further to other benchmarks. As part of that work, we also added the capability in our scripts to use the profiling parameters passed by the user or to set them to default if nothing's specified.
These profiling parameters should be added to the relevant benchmark configs so that user can tune these params according to their needs.
We should add all these parameters mentioned below and anything else that I might have missed:
PROFILING_TOOLPROFILING_JAVA_OPTIONSPROFILING_PROFILE_TIMEPROFILING_SLEEP_TIMEPERF_SAMPLING_PERIODPERF_EVENT
Proposed Work
- [ ] Update benchmark versions
- [ ] Add any new or missing profiling parameters to PerfNext Launcher
Selected jobs are the ones that we care about:
https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/b3510949c0d6f24650f48d393769ec5d10c35c2d/PerfNext/config/benchmarks/data_simple/Liberty.xml#L3 https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/b3510949c0d6f24650f48d393769ec5d10c35c2d/PerfNext/config/benchmarks/data_simple/Liberty.xml#L239 https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/b3510949c0d6f24650f48d393769ec5d10c35c2d/PerfNext/config/benchmarks/data_simple/ODM.xml#L3 https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/b3510949c0d6f24650f48d393769ec5d10c35c2d/PerfNext/config/benchmarks/data_simple/ODM.xml#L27
- [ ] After updating the versions and adding new profiling options, we should test our changes with both ODM and Liberty for all the supported profiling tools for each benchmark. We should make sure to test one (i.e. DT7, any ODM) of the 2 jobs from each benchmark suite extensively (i.e. Run all the 7 currently supported profiling tools as mentioned below). For the other jobs (i.e. DT3, other ODM job), we could do some sanity testing to make sure things work.
Default Testing: Parameters (Avoid setting profiling parameters unless required. You might just need to set PROFILING_TOOL. Leave the non-profiling options as default.)
Non-default Testing: Explicitly set all profiling parameters to some value similar to their default but not the same as default value. For example, if the default value for PROFILING_SLEEP_TIME is 60, test it with 30 or something.
Currently Supported Profiling Tools:
- jprof tprof
- jprof scs
- jprof callflow
- jprof calltree
- jprof rtarcf
- perf stat
- perf record