m3
m3 copied to clipboard
[Experimental][DBnode Client] - Optimize fetch operations by avoiding by deduplicating replicas with identical streams
What this PR does / why we need it: This P.R has the potential to significantly decrease the CPU load of decoding compressed blocks in the M3DB client which is especially important for Protobuf workloads where decoding the stream is very expensive.
Fixes # N.A
Special notes for your reviewer: N.A
Does this PR introduce a user-facing and/or backwards incompatible change?: No this would be a transparent performance increase
Improve M3DB client decoding logic to deduplicate replicas with identical streams on a block-by-block basis, significantly reducing decoding overhead in many cases.
Does this PR require updating code package or user-facing documentation?: N.A
Codecov Report
Merging #2068 into master will increase coverage by
2.4%. The diff coverage is54%.
@@ Coverage Diff @@
## master #2068 +/- ##
========================================
+ Coverage 62.6% 65% +2.4%
========================================
Files 996 1004 +8
Lines 86851 86261 -590
========================================
+ Hits 54386 56119 +1733
+ Misses 28192 25957 -2235
+ Partials 4273 4185 -88
| Flag | Coverage Δ | |
|---|---|---|
| #aggregator | 24% <ø> (-39.3%) |
:arrow_down: |
| #cluster | 77.1% <ø> (-8.5%) |
:arrow_down: |
| #collector | 19.8% <ø> (-29%) |
:arrow_down: |
| #dbnode | 70.1% <54%> (+3.5%) |
:arrow_up: |
| #m3em | 68.4% <ø> (+12.6%) |
:arrow_up: |
| #m3ninx | 70.9% <ø> (+24.1%) |
:arrow_up: |
| #m3nsch | 51.1% <ø> (-48.9%) |
:arrow_down: |
| #metrics | 17.7% <ø> (-82.3%) |
:arrow_down: |
| #msg | 74.7% <ø> (-25.3%) |
:arrow_down: |
| #query | 61.2% <ø> (-8.4%) |
:arrow_down: |
| #x | 77.2% <ø> (+0.8%) |
:arrow_up: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact),ø = not affected,? = missing dataPowered by Codecov. Last update 7e54ccf...3379200. Read the comment docs.
This is cool, let us know if there's any easy numbers to share - would be interesting.
Also wonder if it's worth us sending over checksums to the client so client doesn't have to calculate checksums itself
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
Richard Artoul seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.