Metadata-Filtering not working with Milvus2DocumentStore

[nutrapur]

Describe the bug Metadata filtering does not work with Milvus2DocumentStore and is ignored. Tested in combination with SQLite.

Expected behavior

With the following pipeline only documents with the groups of the filters should be displayed, but the filter is ignored. Other DocumentStores (WeaviateDocumentStore, OpenSearchDocumentStore) behave as expected.

result = pipeline.run(
    query="international climate conferences",
    params={
        "Retriever": {
            "top_k": 10,
            "filters": {"group": ["A", "B"]}
        }
    }
)

FAQ Check

• [x] Have you had a look at our new FAQ page?

System:

• OS: OSX
• GPU/CPU: CPU
• Haystack version (commit or version number): 1.7.0
• DocumentStore: Milvus2DocumentStore
• Retriever: DensePassageRetriever

[TuanaCelik]

@bogdankostic - this issue mentions what you also describe in #2641 - I'm not sure if this is something we are already working on.

[bogdankostic]

Hi @nutrapur, as Tuana pointed out, this is a known issue to us. The reason that filtering is not implemented for Milvus2 is that Milvus currently only allows filtering for scalar data types. One possibility to still have filtering in Milvus2DocumentStore might be via the SQLDocumentStore. This is currently not one of our priorities, so I will put your issue in our backlog.

[masci]

Closing as Milvus support is now external https://github.com/milvus-io/milvus-haystack