dm icon indicating copy to clipboard operation
dm copied to clipboard

add async flush checkpoint feature

Open lance6716 opened this issue 5 years ago • 5 comments

Description

Is your feature request related to a problem? Please describe:

Checkpoint is used when sync binlog to downstream. There're many cases we could advance checkpoint and among them, a case is upstream sending a binlog heartbeat event.

For heartbeat event, currently we wait execution of all DML before it (DM concurrently sync DML), then save the checkpoint. But that will cause the concurrency drop at waiting time.

Describe the feature you'd like:

For heartbeat event, instead of waiting all working goroutine to clear its DML queue, we could find a way like https://github.com/pingcap/dm/pull/605, to smoothly update checkpoint. Note that a checkpoint means all binlog event before it is synced. DM tolerates some event after checkpoint has been synced to downstream by "safe mode".

Score

1800

SIG slack channel

sig-migrate

Mentor

@lance6716

Recommended Skills

How DM sync data

lance6716 avatar Oct 22 '20 11:10 lance6716

/ping

AndreMouche avatar Oct 29 '20 03:10 AndreMouche

pong! I am challenge bot.

ti-challenge-bot[bot] avatar Oct 29 '20 03:10 ti-challenge-bot[bot]

/ping

hidehalo avatar Apr 15 '21 02:04 hidehalo

/pick-up

hidehalo avatar Apr 15 '21 02:04 hidehalo

/pick-up

we're refactoring the ti-challenge-bot, I'll manually add the label for you

lance6716 avatar Apr 15 '21 02:04 lance6716