m3 icon indicating copy to clipboard operation
m3 copied to clipboard

[Experimental][DBnode Client] - Optimize fetch operations by avoiding by deduplicating replicas with identical streams

Open richardartoul opened this issue 5 years ago • 4 comments

What this PR does / why we need it: This P.R has the potential to significantly decrease the CPU load of decoding compressed blocks in the M3DB client which is especially important for Protobuf workloads where decoding the stream is very expensive.

Fixes # N.A

Special notes for your reviewer: N.A

Does this PR introduce a user-facing and/or backwards incompatible change?: No this would be a transparent performance increase

Improve M3DB client decoding logic to deduplicate replicas with identical streams on a block-by-block basis, significantly reducing decoding overhead in many cases.

Does this PR require updating code package or user-facing documentation?: N.A

richardartoul avatar Dec 10 '19 15:12 richardartoul

Codecov Report

Merging #2068 into master will increase coverage by 2.4%. The diff coverage is 54%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #2068     +/-   ##
========================================
+ Coverage    62.6%     65%   +2.4%     
========================================
  Files         996    1004      +8     
  Lines       86851   86261    -590     
========================================
+ Hits        54386   56119   +1733     
+ Misses      28192   25957   -2235     
+ Partials     4273    4185     -88
Flag Coverage Δ
#aggregator 24% <ø> (-39.3%) :arrow_down:
#cluster 77.1% <ø> (-8.5%) :arrow_down:
#collector 19.8% <ø> (-29%) :arrow_down:
#dbnode 70.1% <54%> (+3.5%) :arrow_up:
#m3em 68.4% <ø> (+12.6%) :arrow_up:
#m3ninx 70.9% <ø> (+24.1%) :arrow_up:
#m3nsch 51.1% <ø> (-48.9%) :arrow_down:
#metrics 17.7% <ø> (-82.3%) :arrow_down:
#msg 74.7% <ø> (-25.3%) :arrow_down:
#query 61.2% <ø> (-8.4%) :arrow_down:
#x 77.2% <ø> (+0.8%) :arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 7e54ccf...3379200. Read the comment docs.

codecov[bot] avatar Dec 10 '19 15:12 codecov[bot]

This is cool, let us know if there's any easy numbers to share - would be interesting.

robskillington avatar Dec 14 '19 04:12 robskillington

Also wonder if it's worth us sending over checksums to the client so client doesn't have to calculate checksums itself

robskillington avatar Dec 14 '19 05:12 robskillington

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Richard Artoul seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

CLAassistant avatar May 19 '23 07:05 CLAassistant