trafficserver icon indicating copy to clipboard operation
trafficserver copied to clipboard

Add SO_INCOMING_CPU support to net.sock_option_flag_in

Open masaori335 opened this issue 1 year ago • 0 comments

Address #5165.

Add a option (128) for SO_INCOMING_CPU to the proxy.config.net.sock_option_flag_in.

Background

There was a bug with SO_REUSEPORT since v4.6. But it's fixed by v6.2 and some distros backported it. Now we can use SO_INCOMING_CPU, SO_REUSEPORT and CPU Affinity 4 for perfect locality.

https://github.com/torvalds/linux/commit/b261eda84ec136240a9ca753389853a3a1bccca2

Ref.

    SO_INCOMING_CPU (gettable since Linux 3.19, settable since Linux 4.4)

          Sets or gets the CPU affinity of a socket.  Expects an
          integer flag.

              int cpu = 1;
              setsockopt(fd, SOL_SOCKET, SO_INCOMING_CPU, &cpu,
                         sizeof(cpu));

          Because all of the packets for a single stream (i.e., all
          packets for the same 4-tuple) arrive on the single RX
          queue that is associated with a particular CPU, the
          typical use case is to employ one listening process per RX
          queue, with the incoming flow being handled by a listener
          on the same CPU that is handling the RX queue.  This
          provides optimal NUMA behavior and keeps CPU caches hot.
          

https://www.man7.org/linux/man-pages/man7/socket.7.html

masaori335 avatar May 20 '24 07:05 masaori335