flink-cdc
flink-cdc copied to clipboard
apache
Flink CDC is a streaming data integration tool
This closes FLINK-38806. A post-transform converter will not work if neither projection nor filter rule is defined, as such transform definitions are ignored in TransformTranslator. ```yaml transform: - source-table: foo.bar...
The root cause is that the binlog split metadata transfer protocol relies on the order of finished snapshot split infos to be stable and corresponding to the order of split...
This closes FLINK-38779. CDC Pipeline router supports simple table name replacement for now. A route rule like this: ```yaml - { source-table: 'db.\\.*', sink-table: 'new_db.', replace-symbol: '' } ``` synchronizes...
Transaction BEGIN/END events (when provide.transaction.metadata is enabled) are currently treated as unknown and skipped, causing loss of transaction boundaries. Solution - Detect transaction metadata events via key schema name -...
add MySqlDatabaseSchema gh-ost ddl rules in HistorizedRelationalDatabase
### Description [FLINK-38413](https://issues.apache.org/jira/browse/FLINK-38413): Enables append only table creation from mysql pipeline connector. I have an added an integration test as well that creates out an Append only Paimon table from...
Summary: This PR addresses the issue where the MySQL CDC connector silently fails when encountering reconnection errors (e.g., after credential rotation), ignoring configured `connect.timeout` and `connect.max-retries` parameters and entering an...
This PR adds a new format for defining table properties as `specific.table.properties...`. This allows defining properties that are specific to a table. The table specific property takes precedence. Paimon sink...
https://issues.apache.org/jira/browse/FLINK-38522 # What is the purpose of the change When using StartupOptions.timestamp(), the MySQL CDC connector calls DebeziumUtils.findBinlogOffset() to locate the appropriate binlog position. This method creates a short-lived BinaryLogClient...
Bump jdk version from 8 to 11. This will allow us to gradually start the compatibility work with Flink 2.x, Debezium 2.x, and Iceberg 1.10+.
