bitsail icon indicating copy to clipboard operation
bitsail copied to clipboard

Published 20 hours ago • verified publisher icon bytedance

[BitSail][Connector] Support CDC Source Connector

Open garyli1019 opened this issue 2 years ago • 0 comments

We would like to support a CDC source connector and provide a complete CDC ingestion and integration solution to our user.

We will break this task into 5 stage:

  1. Migrate the legacy Mysql JDBC connector to the V1 interface and refactoring the implementation to include watermark Tracking task: #199
  2. Support Debezium connector to read mysql binlog source.
  3. Implement the snapshot and binlog swap algorithm according to the Netflix DBlog paper
  4. Extend this solution to other database, including oracle mongo postgres sqlserver.
  5. Binlog client performance improvement and advanced feature support(realtime schema evolution, whole database sync e.t.c).

garyli1019 avatar Nov 10 '22 03:11 garyli1019