trafficserver
trafficserver copied to clipboard
Add SO_INCOMING_CPU support to net.sock_option_flag_in
Address #5165.
Add a option (128) for SO_INCOMING_CPU to the proxy.config.net.sock_option_flag_in.
Background
There was a bug with SO_REUSEPORT since v4.6. But it's fixed by v6.2 and some distros backported it. Now we can use SO_INCOMING_CPU, SO_REUSEPORT and CPU Affinity 4 for perfect locality.
https://github.com/torvalds/linux/commit/b261eda84ec136240a9ca753389853a3a1bccca2
Ref.
SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux 4.4)
Sets or gets the CPU affinity of a socket. Expects an
integer flag.
int cpu = 1;
setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu,
sizeof(cpu));
Because all of the packets for a single stream (i.e., all
packets for the same 4-tuple) arrive on the single RX
queue that is associated with a particular CPU, the
typical use case is to employ one listening process per RX
queue, with the incoming flow being handled by a listener
on the same CPU that is handling the RX queue. This
provides optimal NUMA behavior and keeps CPU caches hot.
https://www.man7.org/linux/man-pages/man7/socket.7.html