aqa-test-tools
aqa-test-tools copied to clipboard
Enable PerfNext To Use Openjdk-tests Framework
Background
Currently, PerfNext uses its own pipeline to launch benchmark jobs, a process requiring several setup steps such as downloading and setting up benchmark and SDK packages.
Proposal
PerfNext should use the Openjdk-tests Framework (https://github.com/AdoptOpenJDK/openjdk-tests) by using its pipeline scripts, which have several functionalities including the ones that PerfNext needs. This move would streamline the execution of performance tests into the CI pipelines that are used for other system and functional tests.
Currently, PerfNext has an API called /api/benchengine/submit (aka BenchEngine) that generates the necessary scripts and submits the request to Jenkins.
https://github.com/AdoptOpenJDK/openjdk-test-tools/blob/b528a63ea146080f4bb1430740a734c2af87df7e/PerfNext/app/apis/BenchEngine/parser.js#L42-L54
As shown above, /api/benchengine/submit API currently sends 2 scripts to Jenkins Pipeline:
setupScript: It does all the setup tasks such as downloading benchmark and SDK packages.
benchmarkScript: Exports all the necessary environment variables and runs the main benchmark script.
We could get rid of the setupScript since Openjdk-tests Framework can already do that. We would still need the ability to generate benchmarkScript so that developers are able to change default configs and PerfNext can pass the custom benchmark script to Openjdk-tests Framework to run.
Details
I'll be adding more details to this issue soon.