amplify-js icon indicating copy to clipboard operation
amplify-js copied to clipboard

How to enforce Query instead of Scan on syncQueries

Open Dennis-Dekker opened this issue 1 year ago • 5 comments

Is this related to a new or existing framework?

Vue

Is this related to a new or existing API?

GraphQL API, DataStore

Is this related to another service?

DynamoDB, Appsync

Describe the feature you'd like to request

The documentation of amplify describes how to change the default configuration of data syncing from Scan to Query. It states that the way to do this is to write sync expressions that correspond with the GSIs that are defined in the schema. When testing this functionality we found that syncing the data is still really slow for us, and we suspected the data still to be retrieved by Scans. Further investigation into the documentation of the Sync operations found this piece of text seemingly contradicting the earlier description:

When AWS AppSync receives a request for a Sync operation, it uses the fields specified in the request to determine if the Base table or the Delta table should be accessed.

  • If the lastSync field is not specified, a Scan on the Base table is performed.
  • If the lastSync field is specified, but the value is before the current moment - DeltaSyncTTL, a Scan on the Base table is performed.
  • If the lastSync field is specified, and the value is on or after the current moment - DeltaSyncTTL, a Query on the Delta table is performed.

Only when the lastSync field is specified and the DeltaSyncTTL time limit has not been passed a Query is sent to the Delta table.

Describe the solution you'd like

The documentation suggests that using the correct sync expressions should result in Queries being used. The solution I would like to see (modified copy of the original text):

When AWS AppSync receives a request for a Sync operation, it uses the fields specified in the request to determine if the Base table or the Delta table should be accessed.

  • If the lastSync field is not specified, a Scan or Query on the Base table is performed:
    • If the sync expression corresponds with a GSI or primary index, use a Query.
    • If the sync expression is not specified or does not correspond with a GSI or primary index, use a Scan.
  • If the lastSync field is specified, but the value is before the current moment - DeltaSyncTTL, a Scan or Query on the Base table is performed.
    • If the sync expression corresponds with a GSI or primary index, use a Query.
    • If the sync expression is not specified or does not correspond with a GSI or primary index, use a Scan.
  • If the lastSync field is specified, and the value is on or after the current moment - DeltaSyncTTL, a Query on the Delta table is performed.

Describe alternatives you've considered

One solution we have considered is to copy and modify the generated preAuth resolvers for the syncQueries, implementing the suggested solution above ourselves. This has some drawbacks, most importantly the resolver not being updated automatically when we add a new GSI, missing authentication updates to the resolvers from Amplify and writing the VTL resolvers ourselves (thereby possibly introducing bugs and/or unwanted results).

Additional context

No response

Is this something that you'd be interested in working on?

  • [ ] 👋 I may be able to implement this feature request
  • [ ] ⚠️ This feature might incur a breaking change

Dennis-Dekker avatar Jun 01 '23 12:06 Dennis-Dekker