dapr
Token returned by QueryStateAsync() does not work for subsequent queries
Hello!
I'm using Azure Cosmos DB state store and the QueryStateAsync() method. I want to get all of my data and used var query = "{\"filter\": {}}", but I run into an exception ("state store statestorecosmos query failed: context canceled") if my database has more than 999 rows. I then tried to use pagination, but I'm running into exceptions again with the pagination token returned from Azure Cosmos DB.
Expected Behavior
var query = "{\"filter\": {}, \"page\": {\"limit\": 100}}";
var employeesResponse = await _daprClient.QueryStateAsync<Employee>(STORE_NAME, query, cancellationToken: cancellation);
// employeesResponse contains the first 100 items as expected
var paginationToken = employeesResponse.Token;
query = $"{{\"filter\": {{}}, \"page\": {{\"limit\": 100, \"token\": {paginationToken}}}}}";
employeesResponse = await _daprClient.QueryStateAsync<Employee>(STORE_NAME, query, cancellationToken: cancellation);
// Should return the next 100 items
Actual Behavior
var query = "{\"filter\": {}, \"page\": {\"limit\": 100}}";
var employeesResponse = await _daprClient.QueryStateAsync<Employee>(STORE_NAME, query, cancellationToken: cancellation);
// employeesResponse contains the first 100 items as expected, good so far
var paginationToken = employeesResponse.Token;
// Printing paginationToken outputs {"token":"-RID:~R4EMAJWkwMToAwAAAAAAAA==#RT:1#TRC:1000#ISV:2#IEO:65567#QCF:8","range":{"min":"","max":"FF"}}
query = $"{{\"filter\": {{}}, \"page\": {{\"limit\": 100, \"token\": {paginationToken}}}}}";
employeesResponse = await _daprClient.QueryStateAsync<Employee>(STORE_NAME, query, cancellationToken: cancellation);
// QueryStateAsync() throws an exception
QueryStateAsync() throws an exception with details: state store statestorecosmos query failed: failed to parse JSON query body: json: cannot unmarshal object into Go struct field Pagination.page.token of type string
Modifying the paginationToken to remove the first '{' and last '}' to have "token":"-RID:~R4EMAJWkwMToAwAAAAAAAA==#RT:1#TRC:1000#ISV:2#IEO:65567#QCF:8","range":{"min":"","max":"FF"}
Then modifying the query to query = $"{{\"filter\": {{}}, \"page\": {{\"limit\": 100, {paginationToken}}}}}";
QueryStateAsync() throws an exception with details "Invalid Continuation Token\r\nActivityId: fc7cfc39-cd1d-4e4e-a9d4-7ead0d367bac, Microsoft.Azure.Documents.Common/2.14.0".
I'm using Dapr 1.13.0.
Thanks in advance!
This is probably better asked in dapr/components-contrib as it's unlikely to be specific to the .NET SDK. That said, the Dapr state store query API is considered to be in "alpha" (and has been for some time) and not all components support it (or all aspects of it). The query API is unlikely to make it out of alpha due to changes in priorities as well as other competing proposals for competing functionality.
