opensearch-build
opensearch-build copied to clipboard
[BENCHMARK] Add support for opensearch-benchmark's `compare` feature
Is your feature request related to a problem? Please describe
As of today we are using opensearch-benchmark's execute-test api to run performance benchmark tests on nightly and as well as ad-hoc basis.
We are also adding support to run performance benchmark on OpenSearch pull requests.
So, while a developer is working on their changes, apart from just getting performance benchmark numbers published on the PR, they might also want to know how their changes compare against a baseline.
Describe the solution you'd like
Fortunately, opensearch-benchmark's compare command solves this problem for us. It only requires baseline and contender test-execution-ids as input params and publishes a detailed comparison result.
We need to add this feature into existing benchmark workflow and add a job in jenkins, similar to benchmark-test to execute this workflow on ad-hoc basis and as well as on pull request.
Describe alternatives you've considered
No response
Additional context
No response
Assigning this to @OVI3D0
Commenting for assignment