pelikan icon indicating copy to clipboard operation
pelikan copied to clipboard

WIP: segcache: use mutex and jemalloc

Open brayniac opened this issue 2 years ago • 1 comments

Draft PR, needs performance evaluation

Changes threading model to use shared storage, protected by a mutex. This removes the overhead of cross-thread signaling and communication on the request path in favor of using coarse-locking.

Sets the global allocator to jemalloc for pelikan_segcache_rs

brayniac avatar Sep 16 '22 19:09 brayniac

For a 9 core configuration on: Intel(R) Xeon(R) Platinum 8352V CPU @ 2.10GHz (dual-socket, 72 cores, 144 threads, SMT enabled) 100GbE NIC (E810-C) 72 queue pairs 1 per physical core, NIC local to socket 0 backend threads pinned to a set of cores on socket 0, avoiding NIC IRQs

With a workload: 80:20 Read:Write 4byte keys, 128byte values 1024 connections to backend

With a backend config: 1GB heap, 128KB segment, RandomFifo 7 workers + storage as baseline 8 workers (no storage thread) as experiment

This change brings the redline from 728kqps -> 830kqps

brayniac avatar Sep 19 '22 19:09 brayniac