ElasticVectorSearch constructor to support Elastic Cloud

[brandco]

The current constructor for ElasticVectorSearch does not support connecting to Elastic Cloud.

>>> db = ElasticVectorSearch.from_documents(docs, embeddings,
...                                         cloud_id = CLOUD_ID,
...                                         basic_auth=(USER_NAME, PASSWORD))Traceback (most recent call last):  File "<stdin>", line 1, in <module>
  File "c:\Users\bcollingsworth\code\langchain_testing\.venv\lib\site-packages\langchain\vectorstores\base.py", line 113, in from_documents
    return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)
  File "c:\Users\bcollingsworth\code\langchain_testing\.venv\lib\site-packages\langchain\vectorstores\elastic_vector_search.py", line 161, in from_texts
    elasticsearch_url = get_from_dict_or_env(
  File "c:\Users\bcollingsworth\code\langchain_testing\.venv\lib\site-packages\langchain\utils.py", line 17, in get_from_dict_or_env
    raise ValueError(
ValueError: Did not find elasticsearch_url, please add an environment variable `ELASTICSEARCH_URL` which contains it, or pass  `elasticsearch_url` as a named parameter.

It looks like the current implementation would only support an instance of Elastic Search on localhost without security enabled, which is not recommended.

I can connect with the elasticsearch python client like this:

from elasticsearch import Elasticsearch

es = Elasticsearch(cloud_id=CLOUD_ID, http_auth=(USER_NAME, PASSWORD))
es.info()

See the documentation at Connecting to Elastic Cloud

An alternative constructor to the ElasticVectorSearch that takes a elasticsearch client instance, index name and embedding object would allow the user to deal with all the possible authentication set ups before using langchain.

[sergerdn]

@brandco

The login and password are working as expected, and you can set them up in the elasticsearch_url . Here's a well-known trick:

elasticsearch_url = "https://elastic:YOUR_PASSWORD@es_host:es_port"
elastic_search = ElasticVectorSearch(
        elasticsearch_url= elasticsearch_url,
        index_name= index_name,
        embedding=embedding
)

This approach also applies to ElasticSearch installations on the cloud or any other platform:

elasticsearch_url = "https://elastic:YOUR_PASSWORD@es_host:es_port"
es = Elasticsearch(elasticsearch_url)

es.delete_by_query(index=es_index_name, body={"query": {"match_all": {}}})

[sergerdn]

@brandco, https://github.com/hwchase17/langchain/pull/2141

Does it look good to you, or do you have any further questions? I have documented it in as much detail as possible.

[vibha0411]

I still have an issue. I am trying to use ElasticVectorSearch.from_documents and it is creating a random index name and not using the index_name provided. Is there any reason for this?

[dosubot[bot]]

Hi, @brandco! I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, the issue you reported was about the current constructor for ElasticVectorSearch not supporting connecting to Elastic Cloud, which limits usage to only instances of Elastic Search on localhost without security enabled. One user suggested a workaround by setting up login and password in the elasticsearch_url, and another user provided a pull request with detailed documentation. It seems that the issue has been resolved.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your contribution to the LangChain repository!

[brandco]

Thank you all. I'm sorry I was not able to test this fix because I had to spin down my cloud instance.

[dosubot[bot]]

Thank you, @brandco, for closing the issue! We appreciate your help in keeping LangChain organized and efficient.