opensearch-build icon indicating copy to clipboard operation
opensearch-build copied to clipboard

[BENCHMARK] Add support for opensearch-benchmark's `compare` feature

Open rishabh6788 opened this issue 1 year ago • 2 comments

Is your feature request related to a problem? Please describe

As of today we are using opensearch-benchmark's execute-test api to run performance benchmark tests on nightly and as well as ad-hoc basis. We are also adding support to run performance benchmark on OpenSearch pull requests.

So, while a developer is working on their changes, apart from just getting performance benchmark numbers published on the PR, they might also want to know how their changes compare against a baseline.

Describe the solution you'd like

Fortunately, opensearch-benchmark's compare command solves this problem for us. It only requires baseline and contender test-execution-ids as input params and publishes a detailed comparison result.

We need to add this feature into existing benchmark workflow and add a job in jenkins, similar to benchmark-test to execute this workflow on ad-hoc basis and as well as on pull request.

Describe alternatives you've considered

No response

Additional context

No response

rishabh6788 avatar Jul 10 '24 19:07 rishabh6788

Assigning this to @OVI3D0

rishabh6788 avatar Jul 15 '24 18:07 rishabh6788

Commenting for assignment

OVI3D0 avatar Jul 15 '24 18:07 OVI3D0