flink-cdc
flink-cdc copied to clipboard
apache
Flink CDC is a streaming data integration tool
As the design of [https://github.com/apache/flink-cdc/pull/2220](http://example.com/)，CDC only cares about the maximum GTID position and starts from it. For example, if reading from gtid offset 1-7:9-10, it will automatically adjust to read...
This PR adds Hudi CDC sink support to Flink CDC. As of now the following features are supported: 1. Simple bucket index 2. Non-partitioned tables 3. MOR tables 4. Compaction...
In postgres when a table has a column with timestamp with time zone column the cdc pipeline fails
This PR upgrades the Apache Iceberg dependency from 1.6.1 to 1.10.0 in the iceberg-pipeline-sink connector module. The Iceberg 1.10.0 [release](https://iceberg.apache.org/releases/#1100-release) introduced extended catalog support in the Java Core API, including...
https://github.com/apache/flink-cdc/pull/3529
When using MySQL CDC Pipeline to synchronize data to Paimon, ALTER TABLE MODIFY COLUMN operations that change column order are ignored and not propagated to the downstream sink, causing schema...
WIP, blocked by JDK version bump FLINK-38730
issue: https://issues.apache.org/jira/browse/FLINK-38247 ### Issue An infinite loop occurred when using the **MySqlChunkSplitter** to split a table with a **MySQL BIGINT UNSIGNED primary key**. (This problem happens when the primary key...
Fix potential exception when online schema change happened. Why did this happen? Because when making table schema changes, the original table was not locked. And the online schema change corresponds...
## Introducing AsyncScheduler with PartitionedDeserializationScheduler ### 🚀 Core Architecture **Intelligent Event Routing**: Leverages primary key-based partitioning to distribute data-change events across dedicated single-thread partition workers, ensuring optimal load distribution and...
