qdrant
qdrant copied to clipboard
Support for array values in FieldCondition
Hi! I discovered a potential new feature for the qdrant Filter capabilities.
Problem
In our qdrant instance, we score our custom implementation of the ID (wineId) in the point payload. For example, here's the payload structure:
payload={'adventurous': 0, 'companyId': 5, 'traditional': 1, 'wineId': 'PSOMM-36129206206421651802760151071810121225'}
It's important for our use case to be able to delete qdrant points based on the "wineId". However, I encountered a bottleneck when attempting to implement a bulk delete of these points using our ID. There isn't a way to insert multiple IDs into a single Filter clause.
After a discussion on Discord , I came up with a solution:
conditions = []
# create conditions
for wineId in d['wineId'].values:
conditions.append(
Filter(
must=[
FieldCondition(
key="wineId",
match=MatchValue(value=wineId),
),
FieldCondition(
key="companyId",
match=MatchValue(value=companyId),
),
],
)
)
# run qdrant delete
QDRANT_CLIENT.delete(
collection_name=f"{collectionName}",
points_selector=FilterSelector(
filter=Filter(
should=conditions
)
)
)
In this solution, I generate an array of conditions with all the "wineIds", and then call a Filter with the "should" clause on all these conditions. This solution performs well for us, as it deletes 14k points in approximately 3.5 seconds.
Proposed Solution
My proposal for a new Filter addition is to support the "in / isin" operator, similar to what is supported in MongoDB in. For example, here's how it could be used with the qdrant Python client:
QDRANT_CLIENT.delete(
collection_name=f"{collectionName}",
points_selector=FilterSelector(
filter=Filter(
must=[
FieldCondition(
key="wineId",
# proposed solution
**match=MatchValueIn(value=wineIds)**
),
FieldCondition(
key="companyId",
match=MatchValue(value=companyId),
),
],
)
)
)
)
I understand that this proposal is more focused on improving the code readability, but it would greatly enhance the ease of use and convenience for users.