Maximum throughput should favour UDP over TCP

[BCITMike]

In the sample throughputs, the TCP numbers are higher than UDP numbers. That implies an implementation issue as UDP should almost always have higher throughput than TCP.

Something doesn't seem right for TCP to have higher numbers than UDP.

[ghostop14]

The UDP versus TCP was interesting during testing. In general I lean towards TCP connections, and here's why. I've been a network engineer and done distributed computing development for years and get the TCP/UDP trade-off, but what I noticed and others have run into the same issue too is that with TCP over any modern network switch you get "full speed" throughput whereas if you try to pipe the same data through UDP, you can manage your transmit buffers, but it's all too easy to overflow buffers that are out of your control such as in the switch or receiving system resulting in too much lost data that the sender can't detect. So without building error recovery into a UDP sub-protocol or too much blind throttling to try not to overflow downstream buffers, TCP's just the better way to go and as long as you're in a local switched network, the extra overhead for TCP outweighs the lost UDP packet issue. I tested the results with satellite feeds and decoders over both UDP and TCP in a controlled environment, and watched the dropped UDP packets result in no packet decoding both with the existing gnuradio UDP source blocks and the grnet source versions. Switching to TCP worked fine. If you want to run with UDP, the best way may be to take the UDP-Sink block and connect it to an external program then use the header/crc options that block has. At least that way the receiving program could do some error detection and sequence checking. Hope that helps! I'll leave this open for a bit if you have any thoughts or experience that might help with the udp_source block.

[BCITMike]

TCP is a better way to go when you need reliability and cannot handle packet loss. However, UDP will ALWAYS result in a higher throughput, so when your numbers don't match that, there is something wrong in your setup. For actual data received, UDP must be higher.

UDP is limited by the sending and receiving CPU and the pipe in between. TCP has the same limitations PLUS:

1. overhead
2. backoff delays
3. ramp up transmission rate
4. retransmit lost packets and in flight packets
5. drop in transmission rate when lost packets encountered
6. throughput is decreased by delays in either direction not just in sending side

Buffers are much harder to handle with TCP than UDP (window scaling, etc). With UDP, its a CPU limitation for emptying buffers as fast as CPU can. Generally, that's always higher than the interface like 100Mb or 1Gbe. So when you need peak performance for real time video or audio or just to push maximum traffic, you generally go with UDP over TCP. When you need to recover an exact file, TCP is the way to go.

Let's say your internet connection is 100Mbps. With UDP and a wired network, you should be able to send more traffic than the interface can handle, say 102Mbps, and receive the maximum amount of packets that the 100Mbps interface will allow and drops 2Mbps. Received throughput graph would be a very straight and smooth line at 100Mbps or a hair under. With TCP, traffic will start out slow, ramp up to over 100Mbps, get packet loss, and then ramp down to say 92Mbps and then try 102Mbps, and basically it'll keep increasing/decreasing such that the average throughput is LESS than the 100Mbps and LESS than the UDP pipe. Throughput graph would be rippled in the mid 90Mbps as it searches for the best throughput it can without any errors.

In zero loss transmission, received goodput traffic using UDP will be higher than TCP due to less overhead (amongst other reasons). They will be close. In a lossy transmission, received goodput traffic using UDP will be higher than TCP due to not backing off, retransmitting, or waiting on acknowledgements. The more loss, the more is received over UDP than TCP. TCP can only successfully retransmit losses if the transmission+re-transmits is less than the size of the pipe.

The only times UDP results in less received throughput is when there is a significantly more UDP packets being sent than the pipe can handle and the CPU is insufficient to handle the packet interrupts. You'll often find Intel Nic's being recommended over Realtek Nic's, and its mostly due to interrupt handling and CPU usage, where Intel nic's require less CPU processing. Not sure if that's only because of the NAPI driver or other better things as well.

Example of UDP loss due to CPU. I used to test embedded wireless devices which will have limited pipes with data loss. Using iperf to transmit traffic, you might see behaviour like this if there was only 4Mbps provisioned for the wireless link: Send 3Mbps, receive 3Mbps Send 4Mbps, receive 4Mbps Send 5Mbps, receive 4Mbps Send 6Mbps, receive 4Mbps Send 8Mbps, receive 3.9Mbps Send 10Mbps, receive 3.6Mbps Send 15Mbps, receive 1.8Mbps Meanwhile, sending TCP results in 3.2Mbps and never able to fill the entire pipe due to the over the air delays. To increase TCP throughput, you then need multiple streams to be sent concurrently, and then it might get to 3.8Mbps or something.

It blows my mind that you saw higher throughput with satellite using TCP, as new TCP algorithms had to be made specifically to deal with satellite delay and varies a lot between different operating systems. Talking about throughput, not reliability.

So just to reiterate, UDP traffic should always measure higher than TCP traffic. If it doesn't, something is wrong in the setup.