amazon-documentdb-tools
amazon-documentdb-tools copied to clipboard
Data inconsistency intermittently while using MVU-tool
I tested in my own lab for using MVU-tool. However, when I inserted a amount of data to the source and compare the collection count with the target, there are intermittent data inconsistencies.
From source Docdb, I was enabled the changed stream in specific database.
db.adminCommand({modifyChangeStreams: 1,
database: "lab",
collection: "",
enable: true});
And I running MVU-tool with current start-position.
python3 mvu-cdc-migrator.py \
--source-uri "mongodb://<<username>>:<<password>>@<<Source endpoint>>:27017/?tls=true&tlsCAFile=global-bundle-docdb.pem&replicaSet=rs0&retryWrites=false" \
--target-uri "mongodb://<<username>>:<<password>>@<<target Endpoint>>:27017/?tls=true&tlsCAFile=global-bundle-docdb.pem&replicaSet=rs0&retryWrites=false" \
--start-position 0166d02267000000030100000003000041e2 >> detailed_log2.txt
During migration, intermittent mismatch in the number of source and target data. However, after I killed mvu-cdc-migrator and restarted with same position(0166d02267000000030100000003000041e2), data was synced.
Would you please let me know what the root cause of this data discrepancy is? And would it be possible for mitigate this issue?