milvus-cdc icon indicating copy to clipboard operation
milvus-cdc copied to clipboard

Before creating a Milvus CDC task, if the data between the source cluster and the backup cluster is out of sync, with a data gap of several hours, will the synchronization task still be effective?

Open waitwindy opened this issue 10 months ago • 4 comments

When performing data synchronization between already running clusters, I want to first use the backup tool to synchronize offline data to the backup cluster, and then start a CDC task to synchronize real-time data. Is this approach feasible?

waitwindy avatar Jun 26 '25 10:06 waitwindy

If you have not created a cdc task before, you need to use the backup tool to back up and restore the existing data, and then create a cdc task.

SimFG avatar Jun 26 '25 11:06 SimFG

During the period when backup is restoring the backup cluster, can a CDC task be created successfully if new data arrives at the source cluster?

waitwindy avatar Jun 27 '25 01:06 waitwindy

You need to wait for the backup and restore to complete before creating a task. Otherwise, if the target is imported while the cdc is writing synchronously, data will be out of order.

SimFG avatar Jun 27 '25 07:06 SimFG

You need to wait for the backup and restore to complete before creating a task. Otherwise, if the target is imported while the cdc is writing synchronously, data will be out of order.

What about the data synchronization that occurs between the start of the backup & restore and its completion? How is that handled?

bootstrapupload avatar Oct 14 '25 15:10 bootstrapupload