zilliztech
[Bug]: After cdc synchronization is enabled, the source cluster will not synchronize to the target cluster when creating a new database
Current Behavior
After cdc synchronization is enabled, the source cluster will not synchronize to the target cluster when creating a new database
Expected Behavior
No response
Steps To Reproduce
No response
Environment
milvus: 2.4.20 cdc: milvus-cdc-2.0.0-rc4
Anything else?
No response
Another problem is that we are building a milvus cross cluster disaster recovery system, and using milvus backup and cdc to build a backup cluster. When testing the cdc incremental synchronization, it is found that if data is inserted into the source cluster before the cdc task is created, the data added to the source cluster before the creation cannot be synchronized after the cdc synchronization task is enabled.
You can try the latest main branch. In addition, this feature is currently in the testing phase. more detail: https://github.com/zilliztech/milvus-cdc/issues/188#issuecomment-2709610995
It will be solved later that after the backup synchronization, the data is written before the cdc starts. Is this part of data not synchronized
If you need to back up data, you don't actually need cdc. If you want to create two identical milvus clusters with consistent data, it is recommended to stop writing to source milvus first, then back up the data, restore the data, start cdc, and finally let source milvus work normally.
But this is a production environment. It is impossible to stop the normal use of the source cluster because of the establishment of a backup cluster
This scenario is mainly to establish a disaster recovery cluster. When the old cluster has problems, you can directly switch to the backup cluster. Or cluster migration
You can try the latest main branch. In addition, this feature is currently in the testing phase. more detail: #188 (comment)
HI @SimFG Is this testing completed?
@kish5432 You can try it out, we are currently doing some additional development work