sled
sled copied to clipboard
lock-free radix set for storing page IDs
we're super space inefficient when storing sets of page IDs in the SegmentAccountant
, currently using BTreeSet
s and HashSet
s.
the bitset implementation needs the following properties:
- starts in "grow-only" mode
- after stabilizing the segment, switches to "shrink-only" mode
- will be a u64 with at most 42 bits used
- lock-free adds when in grow-only mode
- adds may be eventually consistent
- lock-free removals when in shrink-only mode
- removals may be eventually consistent
I'm tempted to use a radix trie-based approach instead of something like roaring bitmaps because I don't know how to approach making bitmaps like that lock-free. The eventual consistency tolerance means we can cut a lot of corners where normal lock-free data structures would need to be more responsible for achieving linearizability.
inspiration: