amplify-js icon indicating copy to clipboard operation
amplify-js copied to clipboard

DataStore saves duplicate records when syncing with AutoMerge strategy

Open FranklinRajaGitHub opened this issue 2 years ago • 1 comments

Before opening, please confirm:

JavaScript Framework

Angular

Amplify APIs

DataStore

Amplify Categories

Not applicable

Environment information

System: OS: Windows 10 10.0.19044 CPU: (8) x64 Intel(R) Core(TM) i7-8565U CPU @ 1.80GHz Memory: 2.95 GB / 15.88 GB Binaries: Node: 16.13.2 - C:\Program Files\nodejs\node.EXE npm: 8.8.0 - D:\DSG Projects\Mobile Mini\Projects\Code\physical-inventory-app\node_modules.bin\npm.CMD Browsers: Chrome: 104.0.5112.102 Edge: Spartan (44.19041.1266.0), Chromium (104.0.1293.63) Internet Explorer: 11.0.19041.1566 npmPackages: @angular-devkit/build-angular: ~13.2.3 => 13.2.6 @angular-eslint/builder: ~13.0.1 => 13.0.1 @angular-eslint/eslint-plugin: ~13.0.1 => 13.0.1 @angular-eslint/eslint-plugin-template: ~13.0.1 => 13.0.1 @angular-eslint/template-parser: ~13.0.1 => 13.0.1 @angular/cdk: ^13.3.6 => 13.3.6 @angular/cdk/a11y: undefined () @angular/cdk/accordion: undefined () @angular/cdk/bidi: undefined () @angular/cdk/clipboard: undefined () @angular/cdk/coercion: undefined () @angular/cdk/collections: undefined () @angular/cdk/drag-drop: undefined () @angular/cdk/keycodes: undefined () @angular/cdk/layout: undefined () @angular/cdk/observers: undefined () @angular/cdk/overlay: undefined () @angular/cdk/platform: undefined () @angular/cdk/portal: undefined () @angular/cdk/scrolling: undefined () @angular/cdk/stepper: undefined () @angular/cdk/table: undefined () @angular/cdk/testing: undefined () @angular/cdk/testing/protractor: undefined () @angular/cdk/testing/selenium-webdriver: undefined () @angular/cdk/testing/testbed: undefined () @angular/cdk/text-field: undefined () @angular/cdk/tree: undefined () @angular/cli: ~13.2.3 => 13.2.6 @angular/common: ~13.2.2 => 13.2.7 @angular/common/http: undefined () @angular/common/http/testing: undefined () @angular/common/testing: undefined () @angular/common/upgrade: undefined () @angular/compiler: ~13.2.2 => 13.2.7 @angular/compiler-cli: ~13.2.2 => 13.2.7 @angular/compiler/testing: undefined () @angular/core: ~13.2.2 => 13.2.7 @angular/core/testing: undefined () @angular/forms: ~13.2.2 => 13.2.7 @angular/language-service: ~13.2.2 => 13.2.7 @angular/platform-browser: ~13.2.2 => 13.2.7 @angular/platform-browser-dynamic: ~13.2.2 => 13.2.7 @angular/platform-browser-dynamic/testing: undefined () @angular/platform-browser/animations: undefined () @angular/platform-browser/testing: undefined () @angular/router: ~13.2.2 => 13.2.7 @angular/router/testing: undefined () @angular/router/upgrade: undefined () @aws-amplify/ui-angular: ^2.3.2 => 2.3.2 @aws-amplify/ui-angular/legacy: undefined () @capacitor/android: 3.6.0 => 3.5.1 @capacitor/app: ^1.1.1 => 1.1.1 @capacitor/cli: 3.4.3 => 3.4.3 @capacitor/core: ^3.4.3 => 3.5.1 @capacitor/haptics: 1.1.4 => 1.1.4 @capacitor/keyboard: 1.2.2 => 1.2.2 @capacitor/network: ^1.0.7 => 1.0.7 @capacitor/status-bar: 1.0.8 => 1.0.8 @ionic-native/network: ^5.36.0 => 5.36.0 @ionic/angular: ^6.0.0 => 6.0.13 @ionic/angular-toolkit: ^6.0.0 => 6.1.0 @types/jasmine: ~3.6.0 => 3.6.11 @types/jasminewd2: ~2.0.3 => 2.0.10 @types/lodash: ^4.14.182 => 4.14.182 @types/node: ^12.11.1 => 12.20.47 @typescript-eslint/eslint-plugin: 5.3.0 => 5.3.0 @typescript-eslint/parser: 5.3.0 => 5.3.0 aws-amplify: ^4.3.17 => 4.3.17 cordova-res: ^0.15.4 => 0.15.4 eslint: ^7.6.0 => 7.32.0 eslint-plugin-import: 2.22.1 => 2.22.1 eslint-plugin-jsdoc: 30.7.6 => 30.7.6 eslint-plugin-prefer-arrow: 1.2.2 => 1.2.2 example-typescript: 1.0.0 i: ^0.3.7 => 0.3.7 jasmine-core: ~3.8.0 => 3.8.0 (2.8.0) jasmine-spec-reporter: ~5.0.0 => 5.0.2 karma: ~6.3.2 => 6.3.17 karma-chrome-launcher: ~3.1.0 => 3.1.1 karma-coverage: ~2.0.3 => 2.0.3 karma-coverage-coffee-example: 1.0.0 karma-coverage-istanbul-reporter: ~3.0.2 => 3.0.3 karma-jasmine: ~4.0.0 => 4.0.1 karma-jasmine-html-reporter: ^1.5.0 => 1.7.0 lodash: ^4.17.21 => 4.17.21 memo-parser: 0.2.1 node-example: 1.0.0 npm: ^8.8.0 => 8.8.0 protractor: ~7.0.0 => 7.0.0 protractor-example: 1.0.0 rxjs: ~6.6.0 => 6.6.7 (7.5.5) rxjs/ajax: undefined () rxjs/fetch: undefined () rxjs/internal-compatibility: undefined () rxjs/operators: undefined () rxjs/testing: undefined () rxjs/webSocket: undefined () ts-node: ~8.3.0 => 8.3.0 tslib: ^2.2.0 => 2.3.1 (1.14.1) typescript: ~4.4.4 => 4.4.4 (4.6.2) typescript-example: 1.0.0 zone-mix: undefined () zone-node: undefined () zone-testing: undefined () zone.js: ~0.11.4 => 0.11.5 zone.js/async-test: undefined () zone.js/async-test.min: undefined () zone.js/fake-async-test: undefined () zone.js/fake-async-test.min: undefined () zone.js/jasmine-patch: undefined () zone.js/jasmine-patch.min: undefined () zone.js/long-stack-trace-zone: undefined () zone.js/long-stack-trace-zone.min: undefined () zone.js/mocha-patch: undefined () zone.js/mocha-patch.min: undefined () zone.js/proxy: undefined () zone.js/proxy.min: undefined () zone.js/sync-test: undefined () zone.js/sync-test.min: undefined () zone.js/task-tracking: undefined () zone.js/task-tracking.min: undefined () zone.js/webapis-media-query: undefined () zone.js/webapis-media-query.min: undefined () zone.js/webapis-notification: undefined () zone.js/webapis-notification.min: undefined () zone.js/webapis-rtc-peer-connection: undefined () zone.js/webapis-rtc-peer-connection.min: undefined () zone.js/webapis-shadydom: undefined () zone.js/webapis-shadydom.min: undefined () zone.js/wtf: undefined () zone.js/wtf.min: undefined () zone.js/zone-bluebird: undefined () zone.js/zone-bluebird.min: undefined () zone.js/zone-error: undefined () zone.js/zone-error.min: undefined () zone.js/zone-legacy: undefined () zone.js/zone-legacy.min: undefined () zone.js/zone-patch-canvas: undefined () zone.js/zone-patch-canvas.min: undefined () zone.js/zone-patch-cordova: undefined () zone.js/zone-patch-cordova.min: undefined () zone.js/zone-patch-electron: undefined () zone.js/zone-patch-electron.min: undefined () zone.js/zone-patch-fetch: undefined () zone.js/zone-patch-fetch.min: undefined () zone.js/zone-patch-jsonp: undefined () zone.js/zone-patch-jsonp.min: undefined () zone.js/zone-patch-message-port: undefined () zone.js/zone-patch-message-port.min: undefined () zone.js/zone-patch-promise-test: undefined () zone.js/zone-patch-promise-test.min: undefined () zone.js/zone-patch-resize-observer: undefined () zone.js/zone-patch-resize-observer.min: undefined () zone.js/zone-patch-rxjs: undefined () zone.js/zone-patch-rxjs-fake-async: undefined () zone.js/zone-patch-rxjs-fake-async.min: undefined () zone.js/zone-patch-rxjs.min: undefined () zone.js/zone-patch-socket-io: undefined () zone.js/zone-patch-socket-io.min: undefined () zone.js/zone-patch-user-media: undefined () zone.js/zone-patch-user-media.min: undefined () npmGlobalPackages: @aws-amplify/cli: 9.2.1 ionic: 5.4.16 npm: 8.1.2

Describe the bug

Amplify conflict resolution using Auto-merge

We are developing a Mobile App that uses datastore to sync the data to the cloud into dynamoDB. This is a multi-user app used to count the inventory, So, 2 or more users can count at a time. The requirement is that, when 2 or more users tries to count or add same information from the app by keeping their device in OFFLINE mode, the information gets stored in the local datastore. Later when the users turn the device to ONLINE, that time the data needs to syncs to the DynamoDB table using the Auto-merge conflict resolution strategy.

We have a schema by name Unit

type Unit @model {
    id: ID! @index(name: "byUnit")
    serialNumber: String!
    podName: String!
    stackName: String!
    added: Boolean!
    modified: Boolean!
    plantNumber: String! @index(name: "plantNumber")
    inventoryId: String! @index(name: "inventoryId")
    sequenceNumber: Int!
} 

The issue with this configuration is that, when the data syncs, it doesn't resolve the conflict with duplicate data. It allows duplicate data to be inserted into the table.

Expected behavior

This table should not allow duplicate serial numbers. Expectation is that Auto-merge should table care of this conflict and update it with the most recent data.

Reproduction steps

  1. create a simple ionic angular app
  2. Use amplify CLI to first initialize and use in project.
  3. Add GraphQL api to the app
  4. Congiure it use the default conflict resolution strategy Auto merge.
  5. Update the schema.graphql file with the schema in the below code-snippet.
  6. configure AWS Authentication by following this link: https://docs.amplify.aws/lib/auth/getting-started/q/platform/js/#configure-your-application
  7. Push the changes to aws using the amplify cli commands
  8. User the aws-amplify \ datastore .save to store the information into the dynamoDB.
  9. User DataStore to sync
  10. run ionic serve
  11. Access the app from the browser.
  12. Take it offline, save the data in offline more.
  13. Turn the browser setting to go Online.
  14. The data start synching and can find duplicate records in the Unit table.

Code Snippet

type InventoryUnits @model {
  id: ID!
  companyCode: String!  
  branch: String!
  serialNumber: String!      
  modified: Boolean
  added: Boolean
}

type Users @model {
    id: ID!
    firstName: String!
    lastName: String!
    loginId: String!    
    passcode: String!
    plantId: String!
}

type Pod @model {
    id: ID!
    podName: String!
    stackList: [Stack] @hasMany(indexName: "byStack", fields: ["id"])
    totalUnits: Int!
    plantNumber: String! @index(name: "plantNumber")
    inventoryId: String! @index(name: "inventoryId")
}

type Stack @model {
    id: ID! @index(name: "byStack")
    stackName: String!
    podName: String!
    units: [Unit] @hasMany(indexName: "byUnit", fields: ["id"])
    plantNumber: String! @index(name: "plantNumber")
    inventoryId: String! @index(name: "inventoryId")
}

type Unit @model {
    id: ID! @index(name: "byUnit")
    serialNumber: String!
    podName: String!
    stackName: String!
    added: Boolean!
    modified: Boolean!
    plantNumber: String! @index(name: "plantNumber")
    inventoryId: String! @index(name: "inventoryId")
    sequenceNumber: Int!
}

type InventoryDataDownloadStatus @model {
    id: ID!
    plantNumber: String! @index(name: "plantNumber")
    inventoryId: String! @primaryKey
    downloadStatus: String!
}

Log output

// Put your logs below this line


aws-exports.js

No response

Manual configuration

No response

Additional configuration

No response

Mobile Device

Phone\ tablet

Mobile Operating System

Android

Mobile Browser

edge, chrome or firefox

Mobile Browser Version

No response

Additional information and screenshots

No response

FranklinRajaGitHub avatar Sep 05 '22 14:09 FranklinRajaGitHub

There isn't an easy way to deduplicate identical records with different IDs at this time. The upcoming custom primary key feature will allow users to specify their own primary key fields and method of id generation.

For this particular case the you will be able to use serial number as the primary key then DataStore should handle the auto-merge.

You can follow the progress of the custom primary key feature here: https://github.com/aws-amplify/amplify-js/pull/9432

dpilch avatar Sep 15 '22 15:09 dpilch

Custom primary key has been officially released. Please see our blog: https://aws.amazon.com/blogs/mobile/new-announcing-custom-primary-key-support-for-amplify-datastore/

dpilch avatar Nov 28 '22 15:11 dpilch