flink-clickhouse-sink icon indicating copy to clipboard operation
flink-clickhouse-sink copied to clipboard

Published 20 hours ago • verified publisher icon ivi-ru

log loss

Open yuhailove11 opened this issue 1 year ago • 0 comments

In the incremental synchronization phase, MySQL data is synchronized through Flinkcdc and then inserted into the clickhouse. During the synchronization process, restarting MySQL found that some logs were lost in the clickhouse, and there were no logs in the failed log directory. Through printing the log, it was found that the Flink program had already printed out the lost log of CDC synchronization.

This situation may require multiple restarts of MySQL to replicate I have tested more than ten times, especially when there is a high amount of log insertion and frequent loss of logs

yuhailove11 avatar Apr 12 '23 02:04 yuhailove11