opensearch-benchmark
opensearch-benchmark copied to clipboard
Increase benchmark-metrics-* index shard count
Is your feature request related to a problem? Please describe.
If the same OpenSearch metrics store is used for many tests, the document count can reach the 2B limit/shard.
Describe the solution you'd like
The number of shards used for the metrics index should be increased.
Describe alternatives you've considered
Another alternative is to create the index more frequently than monthly
Posting comment from PR to this issue too:
On second thought, we should revert this change. This change potentially impacts other users since this enforces every user's datastore to use a 10 primary and 1 replicas index regardless of cluster configuration.
I encountered an issue with my personal datastore after this PR was merged in. Prior to the change, my OSB runs were writing metrics to an external OpenSearch datastore. After this change was incorporated, it encountered an issue where the number of shards (10) was not a multiple of the number of awareness attributes (3). For more details, see issue https://github.com/opensearch-project/opensearch-benchmark/issues/330. I could change the cluster settings to get around this issue but users should not have to change their clusters to meet the requirements made by OSB.
Instead of changing the default values as done in this PR, users can override the default number of primaries and replicas with datastore.number_of_replicas
and datastore.number_of_shards
in the datastore config (~/.benchmark/benchmark.ini). This is easy to change and prevents us from messing with the default benchmark-metrics
settings that can potentially interfere with other users' setups.