FrameworkBenchmarks icon indicating copy to clipboard operation
FrameworkBenchmarks copied to clipboard

Minimum response size - new requirement for /plaintext test

Open remittor opened this issue 2 years ago • 0 comments

Problem: Many frameworks use various methods to reduce the size of the response in order to increase the speed.

One way is to populate the Server field with a single character value. There are many examples of this.

Possible Solution: It is worth adding a minimum value for the responses that the server sends to the client in the requirements.

The way to check if this requirement is met is very simple:

$ wrk -t1 -c100 -d15 -H 'Connection: keep-alive' http://127.0.0.1:5000/plaintext -s pipeline.lua -- 16
Running 15s test @ http://127.0.0.1:5000/plaintext
  1 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency    18.70ms   13.56ms  49.70ms   54.60%
    Req/Sec   110.47k    15.71k  141.86k    67.33%
  1648765 requests in 15.00s, 246.86MB read
Requests/sec: 109906.89
Transfer/sec:     16.46MB

246.86MB => 246.86 * 1024 * 1024 = 258851471.36 bytes 258851471.36 / 1648765 = 156.997 bytes

Simple check of the calculated value:

$ curl -w '\n%{size_header} %{size_download}\n' http://127.0.0.1:5000/plaintext
Hello, World!
144 13

144 + 13 = 157 bytes

PROFIT!


In order to prevent a many frameworks from disappearing from the test, such this check should probably be done only when the speed exceeds 4M req/sec.

remittor avatar Mar 23 '23 09:03 remittor