sled icon indicating copy to clipboard operation
sled copied to clipboard

lock-free radix set for storing page IDs

Open spacejam opened this issue 5 years ago • 0 comments

we're super space inefficient when storing sets of page IDs in the SegmentAccountant, currently using BTreeSets and HashSets.

the bitset implementation needs the following properties:

  • starts in "grow-only" mode
  • after stabilizing the segment, switches to "shrink-only" mode
  • will be a u64 with at most 42 bits used
  • lock-free adds when in grow-only mode
  • adds may be eventually consistent
  • lock-free removals when in shrink-only mode
  • removals may be eventually consistent

I'm tempted to use a radix trie-based approach instead of something like roaring bitmaps because I don't know how to approach making bitmaps like that lock-free. The eventual consistency tolerance means we can cut a lot of corners where normal lock-free data structures would need to be more responsible for achieving linearizability.

inspiration:

spacejam avatar Feb 20 '20 12:02 spacejam