Flaky MutationEventExtensionsTest test

[lawmicha]

Will investigate the flaky test https://app.circleci.com/pipelines/github/aws-amplify/amplify-ios/4953/workflows/23e03ca6-e27c-4a90-9f43-7b5aa9c13d93/jobs/42782 testSentModelWithNilVersion_Reconciled

Summarizing some findings on this, so MutationEventExtensionsTest test code tries to set up the state of test using setUpPendingMutationQueue by using a for-loop and saving to storage adapter. A for-loop without waiting for each save result is used to mimic "consecutive saves" scenario. The ordering of saves are performed in order of Save 1, 2, and 3, without waiting for the result of each save. The implementation with the cached formatters could cause the lock to be acquired out of order. ie. Save 1, then 3, then 2. since 2 and 3 are both waiting for the lock while 1 has acquired it. I think this scenario is flaky since acquiring the lock twice causes the out of order scenario to appear more often. By removing the need to acquire the lock twice, the test set-up seems to be consistently successful now- which means the time it takes it iterate over the for loop introduces a delay between saves calls that is greater than the time it takes for calls 2 and 3 to be contending for the lock

Originally posted by @lawmicha in https://github.com/aws-amplify/amplify-ios/issues/1760#issuecomment-1137542608

[lawmicha]

so odd, the data retrieve during set up has inProcess: false but nothing is modifying that flag (created with inProcess: true)

"MutationEvent(id: "A38979B5-16E6-46F1-9051-679D499ACDDD", modelId: "DF0AB521-019D-4E72-B31F-74DD7D25F88B", modelName: "Post", json: "{\"id\":\"DF0AB521-019D-4E72-B31F-74DD7D25F88B\",\"title\":\"title1\",\"content\":\"content1\",\"comments\":[],\"createdAt\":\"2022-05-26T15:54:44.125Z\"}", mutationType: "update", createdAt: Amplify.Temporal.DateTime(foundationDate: 2022-05-26 15:54:44 +0000), version: Optional(1), inProcess: false, graphQLFilterJSON: nil)"

unless there's another process that's picking it up

[lawmicha]

root cause is remote sync engine is deqeuing the mutation events and processing them in parallel with the unit test. most probable solution: don't use the same test base as the others and create a simplified one for testing this, since it's only using storage adapter, we only need an in memory storage adapter to perform these unit tests.

[royjit]

Marked this in the main GH issue -https://github.com/aws-amplify/amplify-swift/issues/2340