flink-cdc
flink-cdc copied to clipboard
[tidb]fix the tidb checkpoints to exactly once.
#2034 I fix that bug.
I used the same method, but there is another problem: when starting from checkpoint, I get an exception occasionally
I used the same method, but there is another problem: when starting from checkpoint, I get an exception occasionally
[why problem appear] I guess this is because the data in tikv was dropped by gc. Therefore, the offset data stored in checkpoints can no longer be found in tikv. If the flink program recovers quickly from checkpoints and the tikv memory is not dropped by gc, no exceptions appear. In contrast, if flink recovers slowly from checkpoints and the tikv memory is dropped by gc, an exception will occur. [how to fix it] I think the solution should be to extend the gc interval of tikv.
Thanks @ArainChen for the great work! Before this PR could be merged, could you please rebase it with latest master
branch?
This pull request has been automatically marked as stale because it has not had recent activity for 60 days. It will be closed in 30 days if no further activity occurs.