components-contrib icon indicating copy to clipboard operation
components-contrib copied to clipboard

Feature Request: CosmosDB State Store Query to support partitioned queries

Open DSpirit opened this issue 2 years ago • 6 comments

Describe the feature

~~When saving data through the DaprClient using the partitionKey metadata, everything is stored fine and is partitioned correctly.~~ When querying data using the DaprClient with a partitionKey metadata Dictionary, It would be great if only the records having the specified partitionKey are returned.

Here's an example setup (and also some screenshots): https://github.com/DSpirit/dapr-cosmosdb-partitioning

Maintainer Edit:

This is not a bug but intended behavior. The description above needs updating so it is clear that this is a feature enhancement request to add support for partitioned queries. Today the CosmosDB State Store Query API implementation intentionally performs cross-partition queries exclusively.

Release Note

RELEASE NOTE: ADD Support for partition queries in CosmosDB StateStore

DSpirit avatar Nov 20 '23 16:11 DSpirit

We use cross-partition queries in our Query API implementation. This is expected behavior. Also note that Query API is Alpha and will not be made stable.

Why are we using cross-partition queries? Because the default value assigned to /partitionKey is the state store key. That means every single entity would have a different partition key value which makes querying impossible.

Dapr's Query API is not a replacement for a CosmosDB SDK to query entities. I also strongly discourage use of the Dapr Query API.

berndverst avatar Nov 21 '23 19:11 berndverst

One possibility we could consider: If the partition key is provided as metadata on the query request, we could perform a regular partitioned query. This however would be a feature request.

If so, can you please update your issue title and description @DSpirit to turn this into a feature request?

berndverst avatar Nov 21 '23 19:11 berndverst

I think @luigirende would be interested in implementing this.

berndverst avatar Nov 21 '23 19:11 berndverst

@berndverst thank you for the update! As we currently use the CosmosDB SDK anyway, it would just be a nice-to-have - so feel free to add it if you have the time :)

DSpirit avatar Nov 22 '23 08:11 DSpirit

@DSpirit created a PR 3227 for this change

luigirende avatar Nov 24 '23 08:11 luigirende

@luigirende wow that was quick! Many thanks :) Looking forward to it!

DSpirit avatar Nov 26 '23 15:11 DSpirit