FrameworkBenchmarks
FrameworkBenchmarks copied to clipboard
Minimum response size - new requirement for /plaintext test
Problem: Many frameworks use various methods to reduce the size of the response in order to increase the speed.
One way is to populate the Server field with a single character value. There are many examples of this.
Possible Solution: It is worth adding a minimum value for the responses that the server sends to the client in the requirements.
The way to check if this requirement is met is very simple:
$ wrk -t1 -c100 -d15 -H 'Connection: keep-alive' http://127.0.0.1:5000/plaintext -s pipeline.lua -- 16
Running 15s test @ http://127.0.0.1:5000/plaintext
1 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 18.70ms 13.56ms 49.70ms 54.60%
Req/Sec 110.47k 15.71k 141.86k 67.33%
1648765 requests in 15.00s, 246.86MB read
Requests/sec: 109906.89
Transfer/sec: 16.46MB
246.86MB => 246.86 * 1024 * 1024 = 258851471.36 bytes 258851471.36 / 1648765 = 156.997 bytes
Simple check of the calculated value:
$ curl -w '\n%{size_header} %{size_download}\n' http://127.0.0.1:5000/plaintext
Hello, World!
144 13
144 + 13 = 157 bytes
PROFIT!
In order to prevent a many frameworks from disappearing from the test, such this check should probably be done only when the speed exceeds 4M req/sec.