[META] Reader/Writer Separation

[mch2]

Please describe the end goal of this project

High level goal is to separate indexing and search traffic within remote store enabled clusters to achieve independent scalability and failure/workload isolation.

Supporting References

https://github.com/opensearch-project/OpenSearch/issues/15237, https://github.com/opensearch-project/OpenSearch/issues/7258 https://github.com/opensearch-project/OpenSearch/issues/14596

Issues

Experimental feature goals - Achieve workload & failure isolation with read scale to zero.

introduce replica type changes, pull based replica, allocation filter and minimal API changes (stats).

• [ ] #15368
• [ ] #15445
• [ ] Updates to restore from snapshot with added search replicas
• [ ] Replication updates to make search replica pull based.
• [ ] Update Segment replication stats APIs and backpressure.

-- Experimental (target 2.17)

GA feature goals: Achieve write scale to zero, ISM/Clients integration, and reasonable (cat/health) API updates.

• [ ] Update recovery for search replicas to be directly from remote store.
• [ ] Update API specs & clients
• [ ] Updates to cat API
• [ ] Updates to cluster health API
• [ ] scale to zero writer changes (TBD)
• [ ] ISM integration to auto scale down writers on index migration

Related component

Search:Performance