tiflow
tiflow copied to clipboard
pingcap
cmd (ticdc): a better Kafka consumer
What problem does this PR solve?
Issue Number: close #10989
What is changed and how it works?
Rewrite Kafka consumer
Check List
Tests
- Unit test
- Integration test
- Manual test (add detailed scripts or steps below)
- No code
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
none
[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please assign charlescheung96 for approval. For more information see the Code Review Process.
The full list of commands accepted by this bot can be found here.
Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment
![ti-chi-bot[bot] avatar](https://avatars.githubusercontent.com/in/214286?v=4 "Published by a pub.dev verified publisher"){kind=small}
Hi @wk989898. Thanks for your PR.
I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
@wk989898: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.
In response to this:
/test all
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
Keep this PR in draft state until it is ready for review and merge. In the meantime, you can use /test cdc-integration-kafka-test to manually trigger the Kafka integration test.
It would be extremely helpful if you could provide a design documentation for this new Kafka consumer. The documentation can includes the drawbacks of the current Kafka consumer, suggestions for improvements, how the new design solves these problems, and why it is better than the old one.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:
Project coverage is 57.6276%. Comparing base (
e3b0bc7) to head (58f71db). Report is 8 commits behind head on master.
Additional details and impacted files
| Components | Coverage Δ | |
|---|---|---|
| cdc | 61.4260% <60.9022%> (-0.0527%) |
:arrow_down: |
| dm | 51.2051% <ø> (+0.0060%) |
:arrow_up: |
| engine | 63.3738% <ø> (-0.0283%) |
:arrow_down: |
| Flag | Coverage Δ | |
|---|---|---|
| unit | 57.6276% <ø> (-0.0218%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
@@ Coverage Diff @@
## master #10990 +/- ##
================================================
- Coverage 57.6493% 57.6276% -0.0218%
================================================
Files 850 850
Lines 126095 126264 +169
================================================
+ Hits 72693 72763 +70
- Misses 48002 48099 +97
- Partials 5400 5402 +2
![codecov[bot] avatar](https://avatars.githubusercontent.com/in/254?v=4 "Published by a pub.dev verified publisher"){kind=small}
PR needs rebase.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.