esnet
iperf3: use incompressible data for streaming test
Various access technologies, as well as transparent WAN accellerators, TCP proxies, VPNs and Encryption protocols willl try to compress (gz) the data stream and/or the packets. This is particularly notable by sending zeroed data payload, or when the data buffer which is used to send, is relatively small (few kB) - this can lead to results of throuhgput which are not attainable with normally compressible data, or incompressible data (e.g. media streaming data).
iperf3 should optionally have the capability to allocate a larger chunk of main memory, and fill this with random data or pre-compressed data (alternatively, consume a file / stdin with user-provided data to send, which is sent repeatedly when the data runs out before the test is over).
Per iprf3 code, the iperf3 packets data is already randomized using "/dev/urandom". However, each message (packet) includes the same contents. I wrote the data into a file using iperf3 -F option and then compressed in using zip. The compression results:
- One packet (1KB / 31KB, 32KB 128KB): 0% compression.
- 4,000 packets of 1KB or 31KB: about 99% compression.
- 4,000 packets of of 32KB: 0% compression.
I am not familiar of the zip compression algorithm but from the above results it seems that it is effective for repeated random values only up to length of 32KB. Do you know what is that length for the network components you mentioned? I.e. what should be the size of the iperf3 "larger chunk of main memory"?
Submitted PR #1907 with proposed solution - allow to send blocks with different randomized data in each (up to certain limit and then repeating the blocks).