opensearch-project
[Remote Store] Add setting to limit primary shards per node for an index / all indices
Is your feature request related to a problem? Please describe
Today we have a setting index.routing.allocation.total_shards_per_node , which can limit maximum number of shards per index/across all indices on a single OpenSearch node. For remote store based clusters, we would also like to limit maximum number of primary shards per index/across all indices . This is because on remote store , only the primaries index the data , while both primaries and replicas serve the search request. Hence ,the primary shards are more CPU heavy , as only they do the indexing .
Describe the solution you'd like
For remote store based clusters, we would also like to limit maximum number of primary shards per index/across all indices .
Related component
Storage:Durability
Describe alternatives you've considered
NA
Additional context
NA
This is a general segment replication concern and not specific to remote store, right?
This is a general segment replication concern and not specific to remote store, right ?
Yes, that is correct Andrew.
@gbbafna There is a separate issue created to https://github.com/opensearch-project/OpenSearch/issues/12250 Do we still need this setting change?
@sachinpkale : rebalancing is not deterministic . This proposed setting will make things quite deterministic. Want to know thoughts of @Arpit-Bandejiya as well on this.
[Storage Triage meeting] We don't see the need for this right now. Will revisit when we see the need in future.