m3 icon indicating copy to clipboard operation
m3 copied to clipboard

[m3ninx] Add support for document compression

Open prateek opened this issue 6 years ago • 2 comments

  • Adds underpinnings to support document data compression in m3ninx.
  • Support for three codecs: snappy, deflate and lz4
  • Minor refactoring to abstract away docs.Reader and docs.Writer type away from the fst.Writer, into the encoding/docs package. It allows the fst.Writer to defer knowledge of the compression internals based on the CompressionOptions knob passed down.

NB: Putting this up despite not being ready to be used in production (as it is currently) to break up PRs. The feature is disabled so landing won't affect any releases. More PRs to follow.

prateek avatar May 19 '19 22:05 prateek

Codecov Report

Merging #1646 (1ce857c) into master (79ee9f5) will decrease coverage by 0.2%. The diff coverage is 59.4%.

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #1646     +/-   ##
========================================
- Coverage    72.3%   72.1%   -0.2%     
========================================
  Files         962     972     +10     
  Lines       80157   80578    +421     
========================================
+ Hits        57967   58169    +202     
- Misses      18398   18582    +184     
- Partials     3792    3827     +35     
Flag Coverage Δ
aggregator 82.4% <ø> (ø)
cluster 85.7% <ø> (ø)
collector 63.9% <ø> (ø)
dbnode 80.0% <ø> (-0.1%) :arrow_down:
m3em 73.2% <ø> (ø)
m3ninx 72.0% <59.4%> (-2.0%) :arrow_down:
m3nsch 51.1% <ø> (ø)
metrics 17.6% <ø> (ø)
msg 74.7% <ø> (ø)
query 67.9% <ø> (ø)
x 86.4% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 79ee9f5...1ce857c. Read the comment docs.

codecov[bot] avatar May 19 '19 22:05 codecov[bot]

Nice(!)

robskillington avatar May 21 '19 22:05 robskillington