DataStore sync events not sent after stop/start when no data present.

[ChadyG]

trafficstars

Describe the bug

We use the DataStore sync events (ready, modelSynced) to know when user data is fully synced to determine if a user is returning (has data) or needs to be set up (no data). Through this, we added calls to stop/start DataStore when a new user is signing up, or a returning user is signing in. Conversely, we clear and start the datastore if the user is swapping between accounts.

When clearing and starting the DataStore, we reliably get all of the initial sync events. However, on stopping and starting the DataStore, we sometimes do not get any sync events. We've found that this seems to occur more specifically on new users with no data, and if the app is returning to the foreground when the DatStore is restarted. Returning to the foreground is a side effect of a user signing up and going to verify their email then returning to the app in our case.

Is it intended behavior for the DataStore sync events to reliably come through after a stop/start or clear/start? If not, we will refactor to not rely on these in the way that we are. Otherwise I will provide more information to diagnose any related problems.

Steps To Reproduce

Steps to reproduce the behavior:
1. Use Auth to sign up a new user with email verification.
2. User sends app to background to verify email.
3. User returns to app.
4. App automatically tries to log user in (check for verification)
5. On successful sign in, app restarts DataStore to ensure initial sync events are fired.
6. Based on some timing, we are not getting sync events.

Expected behavior

DataStore modelSynced and ready events fired after DataStore.stop and start called.

Amplify Framework Version

1.29.3 (also seen on 1.29.1 and 1.29.2)

Amplify Categories

DataStore

Dependency manager

Cocoapods

Swift version

5.0

CLI version

10.7.1

Xcode version

14.2, 14.3

Relevant log output

<details>
<summary>Log Messages</summary>


AmplifyManager:AuthHub.signUpAPI
AmplifyManager: Cleared Datastore
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.networkStatus", context: nil, data: Optional(AmplifyPlugins.NetworkStatusEvent(active: true)))
AmplifyManager: Started Datastore
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.subscriptionEstablished", context: nil, data: nil)
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.syncQueriesStarted", context: nil, data: Optional(AmplifyPlugins.SyncQueriesStartedEvent(models: ["LmClub", "LmSession", "LmUser", "LmDevice", "LmShot", "LmSessionStats", "LmProfile"])))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.modelSynced", context: nil, data: Optional(AmplifyPlugins.ModelSyncedEvent(modelName: "LmClub", isFullSync: true, isDeltaSync: false, added: 0, updated: 0, deleted: 0)))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.modelSynced", context: nil, data: Optional(AmplifyPlugins.ModelSyncedEvent(modelName: "LmDevice", isFullSync: true, isDeltaSync: false, added: 0, updated: 0, deleted: 0)))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.modelSynced", context: nil, data: Optional(AmplifyPlugins.ModelSyncedEvent(modelName: "LmProfile", isFullSync: true, isDeltaSync: false, added: 0, updated: 0, deleted: 0)))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.modelSynced", context: nil, data: Optional(AmplifyPlugins.ModelSyncedEvent(modelName: "LmSession", isFullSync: true, isDeltaSync: false, added: 0, updated: 0, deleted: 0)))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.modelSynced", context: nil, data: Optional(AmplifyPlugins.ModelSyncedEvent(modelName: "LmSessionStats", isFullSync: true, isDeltaSync: false, added: 0, updated: 0, deleted: 0)))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.modelSynced", context: nil, data: Optional(AmplifyPlugins.ModelSyncedEvent(modelName: "LmShot", isFullSync: true, isDeltaSync: false, added: 0, updated: 0, deleted: 0)))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.modelSynced", context: nil, data: Optional(AmplifyPlugins.ModelSyncedEvent(modelName: "LmUser", isFullSync: true, isDeltaSync: false, added: 0, updated: 0, deleted: 0)))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.syncQueriesReady", context: nil, data: nil)
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.syncStarted", context: nil, data: nil)
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.ready", context: nil, data: nil)
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.outboxStatus", context: nil, data: Optional(AmplifyPlugins.OutboxStatusEvent(isEmpty: true)))
AppDelegate: didEnterBackgroundNotification

AppDelegate: willEnterForegroundNotification
AmplifyManager:AuthHub.signedIn
AmplifyManager: Stopped Datastore
AmplifyManager: Started Datastore

AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.subscriptionEstablished", context: nil, data: nil)
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.syncQueriesStarted", context: nil, data: Optional(AmplifyPlugins.SyncQueriesStartedEvent(models: ["LmClub", "LmSession", "LmUser", "LmDevice", "LmShot", "LmSessionStats", "LmProfile"])))

AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.syncStarted", context: nil, data: nil)
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.outboxStatus", context: nil, data: Optional(AmplifyPlugins.OutboxStatusEvent(isEmpty: false)))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.outboxMutationEnqueued", context: nil, data: Optional(AmplifyPlugins.OutboxMutationEvent(modelName: "LmUser", element: AmplifyPlugins.OutboxMutationEvent.OutboxMutationEventElement(model: FullSwingKit.LmUser(id: "158b8dc4-ef03-42af-b826-7ec121501196", setupComplete: true, email: Optional("[email protected]"), phone: nil, fullName: Optional("Chad Godsey"), profileImage: nil, userType: nil, gender: nil, handedness: Optional(FullSwingKit.LmHandedness.right), birthdate: nil, companyName: nil, shippingAddressLine1: nil, shippingAddressLine2: nil, shippingPostcode: nil, shippingLocality: nil, shippingRegion: nil, shippingCountry: nil, subscriptions: nil, createdAt: nil, updatedAt: nil), version: nil, lastChangedAt: nil, deleted: nil))))
AmplifyManager.dataStoreHub HubPayload(eventName: "DataStore.outboxStatus", context: nil, data: Optional(AmplifyPlugins.OutboxStatusEvent(isEmpty: false)))

```

Is this a regression?

No

Regression additional context

Possible, have not verified far back enough.

Device

iPhone 13 Pro Max

iOS Version

iOS 16.3.1, 16.4.1, 16.5

Specific to simulators

No response

Additional context

No response

[harsh62]

@ChadyG Thanks for creating the issue. We will look into it and get back to you as soon as we can.

[lawmicha]

Hi @ChadyG, thank you for the details of your implementation with DataStore.

However, on stopping and starting the DataStore, we sometimes do not get any sync events. We've found that this seems to occur more specifically on new users with no data, and if the app is returning to the foreground when the DatStore is restarted. Returning to the foreground is a side effect of a user signing up and going to verify their email then returning to the app in our case

I believe this is expected if there is no data to be synchronized. However I would expect that the .ready event to always come through. The .modelSynced event may not be emitted if the DataStore implementation performs the sync query, gets no items, and moves onto the next state, without emitting some sort of "modelSynced` event of zero count.

Can you let us know which events are missing? just the modelSynced or both modelSynced's and .ready on this scenario where the user is going from background to foreground. If .ready event is not being emitted, we can try to reproduce with the same scenario and see how we can fix the bug.

If modelSynced event is not being emitted, we can also explore whether it makes sense to emit it with the zero count values. Let us know if .ready event is enough information for your app to continue, without the modelSynced event of zero counts in the payload.

[ChadyG]

Thanks @lawmicha!

After signIn and Datastore stop/start, we get the .syncStarted event, but neither .modelSynced nor .ready are emitted. Our app does watch the .modelSynced events, but not in a way where we require them. We do rely on the .ready event, since we cannot load into the app until the user table is synced (which is synced last due to the internal alphabetical sort on model sync).

[harsh62]

@ChadyG

Apologies for the delayed response. Our team will look into it and provide an update.

[lawmicha]

Hey @ChadyG, thanks for clarifying that modelSynced and ready is not being emitted. I can see in the initial description of the issue that you have the hub payloads after stop/start and it doesn't receive these two events.

Can you try with the latest Amplify version and let us know if you're still seeing the behavior?

Can we get your schema, redacted if needed, and the auth modes?