[BENCHMARK] Add support for opensearch-benchmark's compare
feature
#4837
Labels
enhancement
New Enhancement
compare
feature
#4837
Is your feature request related to a problem? Please describe
As of today we are using opensearch-benchmark's
execute-test
api to run performance benchmark tests on nightly and as well as ad-hoc basis.We are also adding support to run performance benchmark on OpenSearch pull requests.
So, while a developer is working on their changes, apart from just getting performance benchmark numbers published on the PR, they might also want to know how their changes compare against a baseline.
Describe the solution you'd like
Fortunately, opensearch-benchmark's compare command solves this problem for us. It only requires
baseline
andcontender
test-execution-ids
as input params and publishes a detailed comparison result.We need to add this feature into existing benchmark workflow and add a job in jenkins, similar to benchmark-test to execute this workflow on ad-hoc basis and as well as on pull request.
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: