paimon icon indicating copy to clipboard operation
paimon copied to clipboard

[Bug] In cdc database synchronization, data changes to new tables using the combined mode are not monitored, and the job needs to be restarted

Open dsanww opened this issue 1 year ago • 3 comments

Search before asking

  • [X] I searched in the issues and found nothing similar.

Paimon version

paimon-flink-1.18-0.9-20240610.002222-13 paimon-flink-action-0.9-20240610.002222-13

Compute Engine

flink1.18

Minimal reproduce step

In the cdc database synchronization, the combined mode synchronizes new tables, but does not monitor data changes to the new tables, so the job needs to restart

What doesn't meet your expectations?

I want to dynamically listen to the binlog of new table data without restarting the job

Anything else?

No response

Are you willing to submit a PR?

  • [ ] I'm willing to submit a PR!

dsanww avatar Jun 13 '24 03:06 dsanww

I encountered the same issue. I only saw the schema was synchronized, but the data was not synchronized.

xingqing avatar Jul 17 '24 10:07 xingqing

Hi, you should use flink-sql-connector-mysql-cdc-2.3.0.jar. Other versions of this jar still have compatibility bugs with paimon.

discivigour avatar Aug 02 '24 08:08 discivigour

Hi, you should use flink-sql-connector-mysql-cdc-2.3.0.jar. Other versions of this jar still have compatibility bugs with paimon.您好,应使用 flink-sql-connector-mysql-cdc-2.3.0.jar。其他版本的此 jar 还存在与 paimon 兼容性问题的 Bug。

Thank you very much, this is feasible.

I also encountered the same issue: paimon-flink-1.18-0.8.2、flink-sql-connector-mysql-cdc-2.4.2

Pandas886 avatar Aug 14 '24 03:08 Pandas886