flink-cdc
flink-cdc copied to clipboard
[Bug] io/debezium/connector/mysql/MySqlConnection$MySqlConnectionConfiguration.class 类重复导致 异常java.lang.NoSuchMethodError
Search before asking
- [X] I searched in the issues and found nothing similar.
Flink version
1.18.1
Flink CDC version
3.0.1
Database and its version
mysql 5.7
Minimal reproduce step
当启动一个mysql 同步到 mysql的flink 同步任务出现以下异常:
skipUpdateBeforeEvent -> Sink: dd_option_data (1/1)#0 (1c585b0b797021c76b125e4a50cc3922_cbc357ccb763df2852fee8c4fc7d55f2_0_0) switched from INITIALIZING to FAILED with failure cause:
java.lang.NoSuchMethodError: io.debezium.connector.mysql.MySqlConnection$MySqlConnectionConfiguration.<init>(Lio/debezium/config/Configuration;Ljava/util/Properties;)V
at com.ververica.cdc.connectors.mysql.debezium.DebeziumUtils.createMySqlConnection(DebeziumUtils.java:85) ~[flink-connector-mysql-cdc-3.0.1.jar:3.0.1]
at com.ververica.cdc.connectors.mysql.MySqlValidator.createJdbcConnection(MySqlValidator.java:86) ~[flink-connector-mysql-cdc-3.0.1.jar:3.0.1]
at com.ververica.cdc.connectors.mysql.MySqlValidator.validate(MySqlValidator.java:68) ~[flink-connector-mysql-cdc-3.0.1.jar:3.0.1]
at com.ververica.cdc.debezium.DebeziumSourceFunction.open(DebeziumSourceFunction.java:215) ~[flink-connector-debezium-3.0.1.jar:3.0.1]
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:101) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.streaming.runtime.tasks.RegularOperatorChain.initializeStateAndOpenOperators(RegularOperatorChain.java:107) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:753) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.call(StreamTaskActionExecutor.java:100) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreInternal(StreamTask.java:728) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:693) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.runtime.taskmanager.Task.runWithSystemExitMonitoring(Task.java:953) ~[flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.runtime.taskmanager.Task.restoreAndInvoke(Task.java:922) [flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:746) [flink-dist-tis-1.18.1.jar:tis-1.18.1]
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:562) [flink-dist-tis-1.18.1.jar:tis-1.18.1]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
What did you expect to see?
能够正常执行
What did you see instead?
出现以上提及的异常,
Anything else?
原因分析
查看了flink-connector-mysql-cdc 所依赖的类 'MySqlConnection.MySqlConnectionConfiguration'分别在classpath依赖的两个jar包中出现,分别是:
- flink-connector-mysql-cdc-3.0.1:io/debezium/connector/mysql/MySqlConnection$MySqlConnectionConfiguration.class
- debezium-connector-mysql-1.9.7.Final:io/debezium/connector/mysql/MySqlConnection$MySqlConnectionConfiguration.class
明显flink-cdc 为了扩展debezium 的功能,使用拷贝的方式复制了一个同名类,并且在新的类中添加了构造函数,以至于在程序实际运行过程中,java没有加载flink-connector-mysql-cdc-3.0.1包中的MySqlConnectionConfiguration 导致错误:
reference from MySqlConnection document: Copied from Debezium project(1.9.7.final) to add custom jdbc properties in the jdbc url. The new parameter {@code jdbcProperties} in the constructor of {@link MySqlConnectionConfiguration} will be used to generate the jdbc url pattern, and may overwrite the default value.
Are you willing to submit a PR?
- [X] I'm willing to submit a PR!