langchain icon indicating copy to clipboard operation
langchain copied to clipboard

Extend elastic_vector_search.py to allow for kNN indexing/searching

Open jeffvestal opened this issue 1 year ago • 0 comments

Feature request

Extend langchain/embeddings/elasticsearch.py to support kNN indexing and searching. The high-level objectives will be:

  1. Allow for the creation of an index with the correct mapping to store documents including dense_vectors so they can be used for kNN search
  2. Store embeddings in elasticsearch in dense_vector field type
  3. Perform kNN search
  4. Perform Hybrid BM25 (query) + kNN search

Motivation

Elasticsearch support approximate k-nearest neighbor search with dense vectors. The current module only support script score / exact match vector search.

Your contribution

I will work on the code and create the pull request

jeffvestal avatar May 27 '23 17:05 jeffvestal