flink-cdc

flink-cdc copied to clipboard

[oracle]Upgrade oracle-chunk to use rowid sharding to determine imbalance

1

Upgrade and fix the inaccurate and unbalanced judgment of oracle-chunk using rowid sharding

e-miao

[FLINK-35996] Add end-to-end integration test for MySQL to Elasticsearch pipeline

4

This commit introduces comprehensive end-to-end tests for the MySQL to Elasticsearch data pipeline. The new tests cover multiple versions of Flink and Elasticsearch to ensure broad compatibility.

proletarians

[docs][minor] Correct Typos in Documentation/Code

10

This change ensures that the log message is clear and free of typographical errors.

Karl-WangSK

[FLINK-35985][transform] transform rules support substring function

1

Currently, the startIndex of flink-cdc.substr starts from 0, while the startIndex of flink.substr starts from 1, and the startIndex of flink-cdc.substr cannot be less than 0. This is inconsistent with...

MOBIN-F

[FLINK-35762] Cache hashCode for TableID & Transform descriptors

1

This closes FLINK-35762. As previously discussed offline with @lvyanquan, HashMap lookup could be slow since we must calculate key hash code every time, though most key objects are immutable. A...

yuxiqian

[FLINK-36024] Handle BinaryLogClient exceptions when searching for starting binlog offset

2

whhe

This closes FLINK-35980 by: * added test cases for pipeline transform features * corrected usage of `Cache` to avoid ClassLoaderLeakage failure Some cases are ignored for now, until ~~FLINK-35981~~, FLINK-35982,...

yuxiqian

[cdc-connector][mysql]Mysql add numRecordsOutBySnapshot and numRecordsOutByIncremental metrics

3

`numRecordsOutBySnapshot` metrics collects snapshot statistics `numRecordsOutByIncremental` metrics collects incremental statistics `numRecordsOutByIncremental` has three metrics, respectively is `numRecordsOutByIncrementalInsert`, `numRecordsOutByIncrementalUpdate`, `numRecordsOutByIncrementalDelete`  2024.7.12 change metrics name，change `numRecordsOutByIncrementalInsert` to `numRecordsOutByDataChangeEventInsert` 

ChengJie1053

[FLINK-34865][pipeline-connector]support sync table and column comments

6

Currently we don‘t support sync comments of table or column, so the pipeline sink cannot obtain them. Users must get the comments from the source, it's a bad experience.

qg-lin

[FLINK-35968][cdc-connector] Remove dependency of flink-cdc-runtime from flink-cdc-source-connector

3

Current, flink-cdc-source-connectors depends on flink-cdc-runtime, which is not ideal for design and is redundant. This issue is aimed to remove it.

loserwang1024