langchain icon indicating copy to clipboard operation
langchain copied to clipboard

Add ElasticsearchEmbeddings class for generating embeddings using Elasticsearch models

Open jeffvestal opened this issue 1 year ago • 2 comments

This PR introduces a new module, elasticsearch_embeddings.py, which provides a wrapper around Elasticsearch embedding models. The new ElasticsearchEmbeddings class allows users to generate embeddings for documents and query texts using a model deployed in an Elasticsearch cluster.

Main features:

  1. The ElasticsearchEmbeddings class initializes with an Elasticsearch connection object and a model_id, providing an interface to interact with the Elasticsearch ML client through infer_trained_model .
  2. The embed_documents() method generates embeddings for a list of documents, and the embed_query() method generates an embedding for a single query text.
  3. The class supports custom input text field names in case the deployed model expects a different field name than the default text_field.
  4. The implementation is compatible with any model deployed in Elasticsearch that generates embeddings as output.

Benefits:

  1. Simplifies the process of generating embeddings using Elasticsearch models.
  2. Provides a clean and intuitive interface to interact with the Elasticsearch ML client.
  3. Allows users to easily integrate Elasticsearch-generated embeddings.

This is my first PR for this project. I created an integration test file, however, I could use some guidance on how to set it up since it needs an Elasticsearch cluster running an embedding model.

Let me know if there are any structural changes needed or anything missing.

Related issue https://github.com/hwchase17/langchain/issues/3400

jeffvestal avatar Apr 23 '23 18:04 jeffvestal

@jeffvestal would it be possible to add an example notebook to docs/modules/models/text_embedding/examples? can be as simple as the example in the constructor docstring. otherwise LGTM!

dev2049 avatar May 15 '23 20:05 dev2049

@dev2049 Definitely. It might be a couple days as I'm traveling this week but I'll get something in there.

jeffvestal avatar May 15 '23 22:05 jeffvestal