tiflow
tiflow copied to clipboard
TiCDC table span ResolvedTs advancing can be improved
Before asking a question, make sure you have
- [X] Searched existing Stack Overflow questions
- [X] Googled your question
- [X] Searched open and closed GitHub issues
- [X] Read the documentation
What is your question?
Currently, TiCDC uses cdc/puller.Frontier
to advance ResolvedTs. It's based on a heap with some other complex data structures like skiplist
.
For a given number set, heap
is good for such situation: after every time we update one number in the set, we need to calculate top(k)
for them. However, For advancing ResolvedTs, we just need to get top(1)
after a duration, and there could be million updates in the duration. So heap
won't make helpful in this case, just iterate all numbers in the set is fine enough.