osu-web icon indicating copy to clipboard operation
osu-web copied to clipboard

Elasticsearch 8 support

Open notbakaneko opened this issue 2 years ago • 3 comments

Adds support for elasticsearch 8, compatibility added where needed to work with the existing 6.x servers.

  • there is no longer a _doc type in the mapping schema.
  • number_of_replicas in the schema file changed to 0; number_of_replicas can be changed online anyway.
  • Bulk indexing requests in es8 no longer accepts type, 6 requires type, neither seem to need type when querying so there shouldn't need to be any changes here, except;
  • the cross-index terms blacklist filter still needs type to function on es6.
ES version ES_COMPATIBILITY
6 true (default)
7 true or false
8 false

The high_scores indices are not going to be upgraded from 6.x, so no changes there on this side.

This doesn't use ELASTIC_CLIENT_APIVERSIONING as that changes the request header which 6.x doesn't support.


update: es6 support removed

notbakaneko avatar Jul 27 '22 12:07 notbakaneko

What's the es upgrade plan here? Are we going to create a new es instance, reindex everything, and then redeploy pointing to the new instance?

Or if we're going to upgrade existing instance, we can just upgrade current one to 7, update the codes, and then finally upgrade to 8 (and there's no direct upgrade path to 8 anyway).

(and should we finally use clustered es so rolling upgrade is possible?)

nanaya avatar Jul 29 '22 09:07 nanaya

I think updating existing instances to 7 is probably the easiest plan (you're welcome to look at that if you beat me).

peppy avatar Aug 01 '22 04:08 peppy

Production main es instance has been upgraded to 7 so the compat thing can be removed. And might as well update the github action and remove es6 compat for total value in SearchResponse.

nanaya avatar Sep 13 '22 11:09 nanaya