memtable/skiplist: add a purpose-built skiplist
Fixes https://github.com/fjall-rs/lsm-tree/issues/95.
Supersedes #98
With the Borrow<Q> trait bound it's not possible to use InternalKeyRef that would allow to not build an InternalKey (which involves a heap allocation).
Need to use equivalent::Comparable instead.
See: https://github.com/fjall-rs/lsm-tree/pull/98
Todo:
- Evaluate memory overhead vs crossbeam skiplist
- Evaluate read and write latency vs crossbeam skiplist
- Try in a full system (heavily cached) benchmark (https://github.com/marvin-j97/rust-storage-bench) vs crossbeam skiplist
I've tried rebasing this onto the v3 branch.
Outlook now: we don't need a free list because all insertions will be unique (because of unique sequence number).
Removing the const generic N parameter is actually a bit harder than expected because it defines the layout of Buffer<N>; changing it to static makes pointer derefs fail with misaligned pointer dereference.
Makes sense. One day if I find time I'll finish up some of the ideas here.
I deleted the 3.0.0 branch, so this needs to rebased on main now
@ajwerner Can you reopen this PR against main?