haystack icon indicating copy to clipboard operation
haystack copied to clipboard
verified publisher icon deepset-ai

Add support for Elasticsearch 7.16.2 onward

Open masci opened this issue 3 years ago • 2 comments

Is your feature request related to a problem? Please describe. We pin elasticsearch-py to <7.11 but those versions of Elasticsearch are still affected by the log4j vulnerability

Describe the solution you'd like Pin the client to a version that makes me able to run Elasticsearch >=7.16.2 (see https://www.elastic.co/blog/new-elasticsearch-and-logstash-releases-upgrade-apache-log4j2)

masci avatar Aug 19 '22 15:08 masci

We have a draft PR here showing that a few tests are not passing with version 7.17.4: https://github.com/deepset-ai/haystack/pull/2993

julian-risch avatar Aug 22 '22 07:08 julian-risch

The failing tests seem all to be related to OpenSearchDocumentStore. The reason is that the latest compatible version of the elasticsearch-py client with OpenSearch is 7.13.4 (see here). In order to upgrade the Elasticsearch client to a version >= 7.16.2, we need to use the OpenSearch client for the OpenSearchDocumentStore.

bogdankostic avatar Aug 22 '22 16:08 bogdankostic