No Downtime Indexing

[tlovett1]

Add functionality to ensure ElasticPress integration during sync.

[ivankristianto]

I found the issue cannot do search when while doing indexing. And found this related issue. I have think of solution to index/reindex with no downtime:

1. First we create another network alias
2. We do mappings and index all the contents
3. Update the endpoint path to this new alias
4. Destroy the old index

What do you think @tlovett1 ?

[rveitch]

While I think this could be a nice feature, I could see @ivankristianto's suggestion (while clever!) being potentially problematic for sites that use their Elasticsearch cluster directly for other integrations, and when retaining programmatically predicable and reliable index names (without reliance on aliases or proxy handling) is necessary. This could also potentially be problematic depending on the cluster and the size of the data being indexed. I'm not sure we could reliably assume the host cluster may have the necessary provisioned storage to support this, you could probably calculate it with some requests to the cluster api, but it still means some assumptions made on behalf of the user and cluster.

Elasticsearch does support index migration that could be used similarly to your suggestion for a temporary clone of the data, which could allow for the original index to be deleted and reindexed while search is temporarily diverted to the index clone during reindexing, but this depends on the version of Elasticsearch, and also potentially some of the cluster considerations mentioned above.

I believe it is possible to retain search functionality during a reindexing via the ep_is_indexing filter, however the gotcha is that only the currently index data is possible to be queried at that time.

I think this is still potentially doable, but might be best suited as an optional feature (instead of a core function) even if it ships with the core plugin. That's my two cents anyways, and I'm happy to help if needed. :)

[pySilver]

take a look at "alias" functionality of elasticsearch. basically you can have the same index name all the time that just points to different indices in fact. in short

1. setup alias "myalias"
2. point alias to index "myindex"

on update

1. setup index "myindex_yymmddhhii"
2. update alias once new index is "green"
3. keep "myindex", so you can easily revert it.