OpenSearch icon indicating copy to clipboard operation
OpenSearch copied to clipboard

[Searchable Snapshot] Implement an API for creating an index backed by a remote snapshot

Open andrross opened this issue 3 years ago • 0 comments

Implement an API for creating an index backed by a remote snapshot (starting with a naive storage layer to prove out the end-to-end functionality), gated by a feature flag.

A rough sketch of what this should look like exists here: https://github.com/andrross/OpenSearch/pull/102/

There are a few known issues that need to be solved:

  • IndexShard::openEngineAndRecoverFromTranslog() doesn't work properly because there should be no translog for this type of shard. The above branch has a line commented out so a proper solution needs to be developed here.
  • BlobStoreRepository::assertSnapshotOrGenericThread has been commented out
  • IndexShard::createShard() should use the directory factory to create the appropriate directory, as opposed to the hacky branching logic that currently exists
  • The new index settings should be scoped to Property.InternalIndex and validated that this approach will work.

andrross avatar Aug 05 '22 00:08 andrross