luceneserver icon indicating copy to clipboard operation
luceneserver copied to clipboard

Allow S3 for snapshots / backups

Open mikemccand opened this issue 4 years ago • 0 comments

Today in luceneserver, the transferring of shards is entirely node to node communication. This is fast/efficient, but, it does maybe mean in a deep replica case that the primary shard is a hotspot trying to send all these bytes out to the replicas.

We should make this pluggable, and e.g. offer an S3 implementation. This might double as a backup / rollback as well, e.g. if we enable a rolling upgrade, and the upgrade goes badly, and user needs to rollback, if they have the series of snapshots in S3 easily addressable, they could quickly rollback the software version and then pull the right snapshot to recover the index pre-upgrade.

mikemccand avatar Jan 24 '21 00:01 mikemccand