flink-cdc icon indicating copy to clipboard operation
flink-cdc copied to clipboard

Published 20 hours ago • verified publisher icon apache

MySQL CDC incremental snapshot reading will duplicate when a failure happend

Open onlywangyh opened this issue 2 years ago • 3 comments

Describe the bug(Please use English) The source of snapshot phase can support checkpoint in chunk level. If a failure happens, the source can be restored and continue to read chunks from last finished chunks.

But the checkpoint will done multiple times in single split. When the source restored the data will reapeat

Environment :

  • Flink version : 1.13.5
  • Flink CDC version: 2.2
  • Database and version: 5.6

onlywangyh avatar Apr 08 '22 10:04 onlywangyh

I am interested in this issue. Please assign this to me, thanks.

ruanhang1993 avatar Aug 10 '22 02:08 ruanhang1993

I also encountered this problem. when restore from the checkpoint ，the job do all the split。

hezhenghongmail avatar Aug 17 '22 08:08 hezhenghongmail

I also encountered this problem. when restore from the checkpoint ，the job do all the split。

I upgrade the severity to blocker and'll fix this issue soon

leonardBang avatar Aug 18 '22 02:08 leonardBang

Fixed in #1505

leonardBang avatar Nov 10 '22 03:11 leonardBang