network-transport-tcp icon indicating copy to clipboard operation
network-transport-tcp copied to clipboard

TCP_NODELAY=True would be a better default

Open nh2 opened this issue 7 years ago • 1 comments

network-transport-tcp has tcpNoDelay = False by default.

Enabling Nagle's algorithm (not setting TCP_NODELAY) is not appropriate for cases where many multithreaded users multiplex over a single TCP connection, as is done in network-transport-tcp.

As described on e.g. https://eklitzke.org/the-caveats-of-tcp-nodelay:

Multi-threaded programs that share sockets between threads without locks are particularly susceptible to problems related to Nagle’s algorithm. This is because even if the application is structured so that a single thread doesn’t do the write-write-read pattern, if there is one writer thread and another reader thread then the pattern can still manifest. This is one good argument for not sharing sockets between threads without mutexes; usually a better option is to use an event loop.

By now I have seen many performance issues caused by this in real-world uses of network-transport-tcp, e.g.

In my opinion, network-transport-tcp by design needs TCP_NODELAY because it is multiplexing, and thus the actual senders and receivers have no control over in what interleaving messages are sent and received.

Further, it already uses sendMany where possible to combine as much data as possible into a single send, so the additional kernel-space batching performed by Nagle's algorithm does not add much on top (apart from the dreaded 40 ms send delay).

nh2 avatar May 05 '17 13:05 nh2

Python 3.6 made TCP_NODELAY the default for asyncio: https://bugs.python.org/issue27456

nh2 avatar May 13 '17 15:05 nh2

Thank you for your patience! The change has been made and is now part of network-transport-tcp 0.8.4, released today.

LaurentRDC avatar Jul 09 '24 12:07 LaurentRDC

@LaurentRDC Thanks!

nh2 avatar Jul 09 '24 17:07 nh2