ElasticsearchDocumentStore connection fails on Azure Elastic Cloud

[benjaminbluhm]

Describe the bug Thanks for creating this awesome project!

I try to use ElasticsearchDocumentStore via Elastic Cloud on Azure using elastic version v7.17.10.

I will re-produce the problem below.

Error message ConnectionError: Initial connection to Elasticsearch failed. Make sure you run an Elasticsearch instance at `[{'host': 'https://<node-id>.northeurope.azure.elastic-cloud.com', 'port': 9400}]` and that it has finished the initial ramp up (can take > 30s).

Expected behavior Establish connection to elastic cloud

To Reproduce

from haystack.document_stores import ElasticsearchDocumentStore

host = "https://<node-id>.northeurope.azure.elastic-cloud.com"
port = 9400
document_store = ElasticsearchDocumentStore(
   host=host,
   port=port,
   username=ELASTIC_USER,
   password=ELASTIC_PASSWORD,
   index="document"
)

This yields the following error:

ConnectionError: Initial connection to Elasticsearch failed. Make sure you run an Elasticsearch instance at `[{'host': 'https://<node-id>.northeurope.azure.elastic-cloud.com', 'port': 9400}]` and that it has finished the initial ramp up (can take > 30s).

I am able to connect using Elasticsearch client as follows:

from elasticsearch import Elasticsearch

client = Elasticsearch(
    hosts=host,
    http_auth=(ELASTIC_USER, ELASTIC_PASSWORD),
)

I believe prepare_hosts function may cause the issue - it will return the following output:

hosts = [{'host': 'https://<node-id>.northeurope.azure.elastic-cloud.com', 'port': 9400}]

Now when I try to create the client using this hosts object:

client = Elasticsearch(
    hosts=hosts,
    http_auth=(ELASTIC_USER, ELASTIC_PASSWORD),
)

it throws the following error:

ConnectionError: ConnectionError(: Failed to establish a new connection: [Errno -2] Name or service not known) caused by: NewConnectionError(: Failed to establish a new connection: [Errno -2] Name or service not known)

I guess this might be the root cause of the ElasticsearchDocumentStore connection issue reported above.

I am quite new to haystack so it's certainly possible that I am missing something, hope there is a simple solution to this :-)

FAQ Check

• [x] Have you had a look at our new FAQ page?