ceph-nvmeof icon indicating copy to clipboard operation
ceph-nvmeof copied to clipboard

cpu affinity should be supported for the threads of io_context_pool and msgr-worker-*

Open liuqinfei opened this issue 2 years ago • 3 comments

Hi. I try to deploy the ceph-nvmeof and do some performance tests. I find that setting the cpu affinity for the threads of io_context_pool and msgr-worker-* can contribute to much higher performance both for the aarch64/x86 machines . Further more, for aarch64 machines, the performance result showes that deploying the reactor threads and the io_context_pool threads in the same ccl (core cluster, usually 4 cores/ccl) will get better performance that in different ccl. So, implementing this by default and give a parameter for choice in the ceph-nvmeof.conf may help users to get better performance.

liuqinfei avatar Sep 20 '22 13:09 liuqinfei

Thanks for bringing this up. We already have an internal work item on this issue. We plan to support a CPU mask when creating a Ceph cluster context in SPDK.

PepperJo avatar Sep 20 '22 14:09 PepperJo

https://review.spdk.io/gerrit/c/spdk/spdk/+/16451

epuertat avatar May 26 '23 12:05 epuertat

The patch has been merged in SPDK but work is required to expose it through the GW.

PepperJo avatar Jun 30 '23 15:06 PepperJo