network-transport-tcp
network-transport-tcp copied to clipboard
TCP_NODELAY=True would be a better default
network-transport-tcp has tcpNoDelay = False
by default.
Enabling Nagle's algorithm (not setting TCP_NODELAY
) is not appropriate for cases where many multithreaded users multiplex over a single TCP connection, as is done in network-transport-tcp.
As described on e.g. https://eklitzke.org/the-caveats-of-tcp-nodelay:
Multi-threaded programs that share sockets between threads without locks are particularly susceptible to problems related to Nagle’s algorithm. This is because even if the application is structured so that a single thread doesn’t do the write-write-read pattern, if there is one writer thread and another reader thread then the pattern can still manifest. This is one good argument for not sharing sockets between threads without mutexes; usually a better option is to use an event loop.
By now I have seen many performance issues caused by this in real-world uses of network-transport-tcp, e.g.
-
Not being able to do more than 20
spawn
calls per second even on localhost netowrking - slowdowns in
cardano-sl
In my opinion, network-transport-tcp by design needs TCP_NODELAY
because it is multiplexing, and thus the actual senders and receivers have no control over in what interleaving messages are sent and received.
Further, it already uses sendMany
where possible to combine as much data as possible into a single send
, so the additional kernel-space batching performed by Nagle's algorithm does not add much on top (apart from the dreaded 40 ms send delay).
Python 3.6 made TCP_NODELAY
the default for asyncio
: https://bugs.python.org/issue27456
Thank you for your patience! The change has been made and is now part of network-transport-tcp 0.8.4, released today.
@LaurentRDC Thanks!