m3 icon indicating copy to clipboard operation
m3 copied to clipboard

[dbnode] Add index snapshotting

Open notbdu opened this issue 4 years ago • 2 comments

What this PR does / why we need it:

Adds index snapshotting. This will reduce the amount of bootstrapping time since we won't need to build the entire index segment.

Also implements initial work for eagerly offloading terminal or frozen index segments from memory.

Special notes for your reviewer:

Does this PR introduce a user-facing and/or backwards incompatible change?:

NONE

Does this PR require updating code package or user-facing documentation?:

NONE

notbdu avatar Aug 26 '20 04:08 notbdu

Codecov Report

Merging #2556 (ed594f5) into master (2ac4853) will decrease coverage by 0.1%. Report is 1044 commits behind head on master. The diff coverage is 75.3%.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #2556     +/-   ##
========================================
- Coverage    72.1%   72.1%   -0.1%     
========================================
  Files        1099    1099             
  Lines       99581   99901    +320     
========================================
+ Hits        71863   72079    +216     
- Misses      22807   22885     +78     
- Partials     4911    4937     +26     
Flag Coverage Δ
aggregator 75.7% <ø> (-0.1%) :arrow_down:
cluster 84.9% <ø> (ø)
collector 84.3% <ø> (ø)
dbnode 79.3% <76.8%> (-0.1%) :arrow_down:
m3em 74.4% <ø> (ø)
m3ninx 73.1% <27.2%> (-0.1%) :arrow_down:
metrics 17.2% <ø> (ø)
msg 74.2% <ø> (+0.1%) :arrow_up:
query 69.0% <ø> (ø)
x 80.1% <ø> (-0.1%) :arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 2ac4853...ed594f5. Read the comment docs.

codecov[bot] avatar Oct 09 '20 21:10 codecov[bot]

Note: With new transparent indexing, LoadBlock should not transparently index when this lands. Also, would be good to have in documentation the snapshot metadata files and how they all relate to each other.

robskillington avatar Oct 16 '20 16:10 robskillington