flink-cdc icon indicating copy to clipboard operation
flink-cdc copied to clipboard

[mysql] Fix the binlogSplit state incompatibility when upgrading from 2.2.0

Open lzshlzsh opened this issue 2 years ago • 3 comments

EOFException will happen when upgrading from mysql-cdc 2.2.0 and restoring from a savepoint. The exception stack is ac follows: java.io.EOFException at org.apache.flink.core.memory.DataInputDeserializer.readBoolean(DataInputDeserializer.java:125) at com.ververica.cdc.connectors.mysql.source.split.MySqlSplitSerializer.deserializeSplit(MySqlSplitSerializer.java:165) at com.ververica.cdc.connectors.mysql.source.split.MySqlSplitSerializer.deserialize(MySqlSplitSerializer.java:124)

The reason is that isSuspended flag is serialized for MySqlBinlogSplit in #996 which results in state incompatibility and we should increase serializer version normally, but we did not.

We can fix the problem simply by checking whether there is data left in buffer as isSuspended flag is the last field.

lzshlzsh avatar May 22 '22 11:05 lzshlzsh

Hi @lzshlzsh, sorry for the delay of this PR. Could you please rebase it to latest master branch since there’s been lots of changes in Flink CDC repo since your original commit? Kindly reminder that com.ververica.cdc.connectors.mysql package has been moved to org.apache.flink.cdc.connectors.mysql.

yuxiqian avatar Apr 25 '24 07:04 yuxiqian