Add support for version-based subscription conflict resolution

[bwerther]

Describe the bug I've been trying out Delta Sync with a test iOS client and I've been running into a problem. If there are 2 or 3 mutations of the same object in a short period of time, AppSync subscription doesn't guarantee the order that the associated subscriptions will arrive in. On the server side we can enforce ordering and apply a sequence number, but on the client side Delta Sync caches the events in the order they arrive and there doesn't seem to be any way to reject events with an older sequence number than we have saved. The result is that sometimes the client overwrites the event with the latest sequence number and shows data that is a sequence number or two out of date.

To Reproduce Steps to reproduce the behavior:

1. Start with any schema that allows update mutations and has a subscription on them. My test has a list of counters that can be incremented or decremented via up/down buttons on each.
2. Setup Delta Sync on iOS against that schema. (I'm using a base query and subscription query but no delta query here)
3. Generate 2-3 mutations in rapid succession on an object.
4. If you do #3 half a dozen times, you'll probably see at least one case where the update subscription returns out of order and you see the wrong data (i.e. from one or two events earlier)

Expected behavior I would expect some mechanism or hooks in DeltaSync to allow us to re-establish subscription event ordering on the client side

Environment(please complete the following information):

• AppSync SDK Version: [2.7.0]
• Dependency Manager: [Cocoapods]
• Swift Version : [4.2]

[bwerther]

Btw, I'm seeing this problem frequently -- e.g. when a client has offline mutations and reconnects, they are played back quickly to all other subscribing clients. Quite frequently they end up out of order and one or more of the clients shows the wrong data. This seems like a pretty fundamental flaw in the whole Delta Sync model unless there is a fix or best-practice that I'm missing to avoid the issue.

[frankmuellr]

Hi @bwerther,

Thank you for raising the issue. We are investigating and will report back.

[palpatim]

Hi @bwerther,

We do guarantee the delivery order of mutations to the server, and as you say it is possible to do some consistency checks on the server, but it's uncommon to have client-enforced ordering of server-received events--at least, it's not a pattern I've encountered frequently. It raises a whole set of synchronization questions when events are received from sources other than the app itself, since the app wouldn't know to wait for the externally-sourced event.

Am I missing or misunderstanding a characteristic of your problem that makes this problem different from what I've described?

[bwerther]

Thanks Tim. The scenario seems like a pretty common one — multiple mutations happen against an object, and all subscribers should quickly converge to a common view of that object. This doesn’t seem to be the case here — I.e. the server holds the definitive view of the object, but subscribers may have played the mutation messages out of order and see out-of-date data (without any expectation of converging to consistency until the object is reloaded). This is likely to happen whenever mutations occur close in time, especially when re-syncing after being offline.

Most protocols I can imagine here would use a sequence number to ensure mutations are applied to the local cache in a strictly increasing order. Without this some number of clients will just have wrong data frequently, which seems like a non-starter to me. Am I missing something here?

On Thu, Jan 3, 2019 at 08:56 Tim Schmelter [email protected] wrote:

Hi @bwerther https://github.com/bwerther,

We do guarantee the delivery order of mutations to the server, and as you say it is possible to do some consistency checks on the server, but it's uncommon to have client-enforced ordering of server-received events--at least, it's not a pattern I've encountered frequently. It raises a whole set of synchronization questions when events are received from sources other than the app itself, since the app wouldn't know to wait for the externally-sourced event.

Am I missing or misunderstanding a characteristic of your problem that makes this problem different from what I've described?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/awslabs/aws-mobile-appsync-sdk-ios/issues/114#issuecomment-451206035, or mute the thread https://github.com/notifications/unsubscribe-auth/AA0MVk9CYtqOGT2JA7JXSxzs-tpDplIeks5u_jZWgaJpZM4Y7QQg .

[palpatim]

Hi @bwerther,

Thanks for clarifying. I'll take this on as a feature request to the team. At the minimum, the SDK could provide a notification to the calling code that a subscription was received on an object whose version was out-of-order. That would at least give app code the ability to designate its desired conflict resolution (similar to how we do conflict resolution on mutations currently), and tell AppSync to proceed with the object merge, or discard it.

We'll update this thread when we have more information.

[atierian]

Thank you for opening this issue. AWS AppSync SDK for iOS entered maintenance mode in September 2023 and will receive no further updates as of September 2024.

Please use Amplify Swift going forward. For information on upgrading to Amplify Swift, refer to the Upgrade from AppSync SDK documentation.