tiflow
tiflow copied to clipboard
mounter(ticdc): optimize mounter cpu usage by reduce memory allocation on map and slice
What problem does this PR solve?
Issue Number: close #11210
What is changed and how it works?
- use
sync.Pool
to reduce the map allocation for the datum - remove
columnInfo
slice from thedatums2Column
method, to reduce memory allocation.
Check List
Tests
- Unit test
- Integration test
Questions
Will it cause performance regression or break compatibility?
Do you need to update user documentation, design documentation or monitoring documentation?
Release note
`None`
Codecov Report
Attention: Patch coverage is 92.15686%
with 4 lines
in your changes missing coverage. Please review.
Project coverage is 57.6113%. Comparing base (
7fbae79
) to head (57b01d9
). Report is 18 commits behind head on master.
:exclamation: Current head 57b01d9 differs from pull request most recent head f59703a
Please upload reports for the commit f59703a to get more accurate results.
Additional details and impacted files
Components | Coverage Δ | |
---|---|---|
cdc | 61.4317% <92.1568%> (-0.0518%) |
:arrow_down: |
dm | 51.1638% <ø> (-0.0424%) |
:arrow_down: |
engine | 63.4020% <ø> (+0.0494%) |
:arrow_up: |
Flag | Coverage Δ | |
---|---|---|
unit | 57.6113% <92.1568%> (-0.0363%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
@@ Coverage Diff @@
## master #11081 +/- ##
================================================
- Coverage 57.6475% 57.6113% -0.0363%
================================================
Files 850 850
Lines 126040 126055 +15
================================================
- Hits 72659 72622 -37
- Misses 47985 48026 +41
- Partials 5396 5407 +11
Please add the test result.
Please add the test result.
The metrics is lost, since I use continous profiling which delete metrics data each 3 hours.
On the master branch, we can see obvious makemap
called inside the decodedRowV2
method, and makeslice
around 2.0% CPU usage inside the datum2Columns
method.
In this PR, the makemap
cannot be seen, and makeslice
reduced to around 1.1%.
[LGTM Timeline notifier]
Timeline:
-
2024-05-31 08:04:22.765480925 +0000 UTC m=+3022816.522616496
: :ballot_box_with_check: agreed by CharlesCheung96.
Could you please provide a benchmark test result? It would be really helpful.
The metrics is lost, since I use continous profiling which delete metrics data each 3 hours.
On the master branch, we can see obvious
makemap
called inside thedecodedRowV2
method, andmakeslice
around 2.0% CPU usage inside thedatum2Columns
method.In this PR, the
makemap
cannot be seen, andmakeslice
reduced to around 1.1%.
The metrics is lost, since I use continous profiling which delete metrics data each 3 hours.
On the master branch, we can see obvious makemap called inside the decodedRowV2 method, and makeslice around 2.0% CPU usage inside the datum2Columns method.
In this PR, the makemap cannot be seen, and makeslice reduced to around 1.1%.
[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: asddongmen, CharlesCheung96
The full list of commands accepted by this bot can be found here.
The pull request process is described here
- ~~OWNERS~~ [CharlesCheung96,asddongmen]
Approvers can indicate their approval by writing /approve
in a comment
Approvers can cancel approval by writing /approve cancel
in a comment
@3AceShowHand: The following tests failed, say /retest
to rerun all failed tests or /retest-required
to rerun all mandatory failed tests:
Test name | Commit | Details | Required | Rerun command |
---|---|---|---|---|
pull-cdc-integration-pulsar-test | f59703ae3c583e094370068a597537791092805f | link | unknown | /test cdc-integration-pulsar-test |
jenkins-ticdc/verify | f59703ae3c583e094370068a597537791092805f | link | unknown | /test verify |
pull-cdc-integration-mysql-test | f59703ae3c583e094370068a597537791092805f | link | unknown | /test cdc-integration-mysql-test |
Full PR test history. Your PR dashboard.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.
PR needs rebase.
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.