haystack icon indicating copy to clipboard operation
haystack copied to clipboard

FAISS in OpenSearch: Support HNSW for cosine

Open tstadel opened this issue 2 years ago • 0 comments

As a user I want to use FAISS's HNSW for cosine similarity too with OpenSearchDocumentStore.

According to https://opensearch.org/docs/latest/search-plugins/knn/knn-index/ FAISS plug-in only supports dot product and l2. So we have to normalize vectors before indexing/searching.

  • [ ] normalize vectors manually at index time
  • [ ] normalize vectors manually at query time
  • [ ] use dot_product space under the hood

tstadel avatar Jul 28 '22 16:07 tstadel