pelikan
pelikan copied to clipboard
WIP: segcache: use mutex and jemalloc
Draft PR, needs performance evaluation
Changes threading model to use shared storage, protected by a mutex. This removes the overhead of cross-thread signaling and communication on the request path in favor of using coarse-locking.
Sets the global allocator to jemalloc for pelikan_segcache_rs
For a 9 core configuration on: Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz (dual-socket, 72 cores, 144 threads, SMT enabled) 100GbE NIC (E810-C) 72 queue pairs 1 per physical core, NIC local to socket 0 backend threads pinned to a set of cores on socket 0, avoiding NIC IRQs
With a workload: 80:20 Read:Write 4byte keys, 128byte values 1024 connections to backend
With a backend config: 1GB heap, 128KB segment, RandomFifo 7 workers + storage as baseline 8 workers (no storage thread) as experiment
This change brings the redline from 728kqps -> 830kqps