haystack icon indicating copy to clipboard operation
haystack copied to clipboard

Enable filtering for FAISSDocumentStore with query_by_embedding()

Open TuanaCelik opened this issue 2 years ago • 2 comments

Currently there is an explicit check in FAISSDocumentStore that will log a warning when using filters. Here is the line of code

This means you can't run a pipeline with filters if you're using FAISSDocumentStore

This check is implemented in the query_by_embedding() function.

Solution Implement filtering for FAISSDocumentStore 😊

Reported by @Zeni69

TuanaCelik avatar Aug 11 '22 15:08 TuanaCelik

Adding here as a note: FAISS doesn't natively support filtering. Here we can find two possible workarounds to this problem, which will probably be quite inefficient, such that in any case it would probably more advisable to use a DocumentStore that natively supports filtering, like OpenSearchDocumentStore or WeaviateDocumentStore.

bogdankostic avatar Aug 17 '22 14:08 bogdankostic

I add a useful resource to understand better this topic: The Missing WHERE Clause in Vector Search by @jamescalam.

anakin87 avatar Sep 09 '22 12:09 anakin87

Hi, Is there any update on filtering with FAISS Documentstore

vijayirlapati avatar Oct 28 '22 11:10 vijayirlapati