[DOC] Guidance about how to size the indices/shards

[fatcloud233]

What do you want to do?

• [ ] Request a change to existing documentation
• [x] Add new documentation
• [ ] Report a technical problem with the documentation
• [ ] Other

Tell us about your request.

Hi, I’m working an project that may introduce a large number of indices into an OpenSearch cluster. I found that for ElasticSearch, there is a good doc: Size Your Shards, which gives some recommendations on the max number of indices/shards per GB heap. I wondering if there is similar recommendation for OpenSearch? And there is big update in ElasticSearch 8.3, which seems having big improvement on max shard counts, see this doc change commit: https://github.com/elastic/elasticsearch/pull/86223. Is there similar effort in OpenSearch? Basically I just want to see what’s the practice for OpenSearch. I think this will be helpful for others too.

Do you have any related resources to help us get started?

This is the ElasticSearch 8.3 doc: https://www.elastic.co/guide/en/elasticsearch/reference/8.3/size-your-shards.html

Is there anything else you'd like to add?

[cwillum]

@fatcloud233 Thanks for adding this request and including the detail. The team will review it soon. In the meantime, you might want to have a look at this document for OpenSearch Services: Sizing Amazon OpenSearch Service domains. Considering your current project's requirements, I'd be curious to know if you find anything else helpful in it.