lantern icon indicating copy to clipboard operation
lantern copied to clipboard

Replace linkedlist for takenbuffer with fixed array

Open var77 opened this issue 1 year ago • 2 comments

  • Replace takenbuffers linked list with fixed size (m*2) array and release unused buffers when pinning new ones after the array is filled.
  • Remove unused htab and fa_cache instances in retriever context
  • Only lock buffers when getting nodes if LANTERNDB_COPYNODES is enabled

TODO

  • fix LANTERNDB_COPYNODES path, currently tests are failing with incorrect scan results

var77 avatar Oct 07 '24 10:10 var77

Benchmarks

metric old new pct change
recall (after create) 0.951 0.943 -0.84%
recall (after insert) 0.000 0.000 -
select tps 22287.227 25784.272 +15.69%
select bulk(100) tps 34.705 39.108 +12.69%
select latency (ms) 0.792 ± 1.649𝜎 0.818 ± 1.760𝜎 +3.28%
select bulk(100) latency (ms) 909.948 ± 118.369𝜎 789.290 ± 113.334𝜎 -13.26%
create latency (ms) 401926.841 401222.500 -0.18%
insert tps 438.335 541.854 +23.62%
insert bulk(100) tps 4.551 5.537 +21.67%
insert latency (ms) 72.304 ± 15.802𝜎 58.199 ± 13.234𝜎 -19.51%
insert bulk(100) latency (ms) 6844.376 ± 193.644𝜎 5654.672 ± 112.456𝜎 -17.38%
disk usage (bytes) 8192008192.000 8192008192.000 -

github-actions[bot] avatar Oct 07 '24 10:10 github-actions[bot]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

:loudspeaker: Thoughts on this report? Let us know!

codecov[bot] avatar Oct 07 '24 10:10 codecov[bot]