Same rate, different connections count - RS times greatly differs - my wrong setup?

[gmiejski]

Have simple http endpoint, I'm the only person using it (no noise from other users - did tests several times). What I notice is that running 2 tests with same rate, but different client connections makes significant change in response times.

If number of connections would be too low, I would assume the rate requirement not fulfilled.

Maybe it has something to do with thread calibrating - I know proper performance testing is hard, but cannot find a guide of how to setup parameters (threads, rate, connections) and how it's connected with Thread calibrating - anyone knows what I'm doing wrong?

Details:

When I run this: wrk -t5 -c5 -d1m -R10 --latency -s ./my_script.lua http://somewhere/something I get this:

  5 threads and 5 connections
  Thread calibration: mean lat.: 197.251ms, rate sampling interval: 524ms
  Thread calibration: mean lat.: 196.915ms, rate sampling interval: 494ms
  Thread calibration: mean lat.: 204.040ms, rate sampling interval: 598ms
  Thread calibration: mean lat.: 181.220ms, rate sampling interval: 448ms
  Thread calibration: mean lat.: 191.452ms, rate sampling interval: 474ms
...
 50.000%  189.95ms
 75.000%  228.61ms
 90.000%  263.93ms
 99.000%  340.99ms
 99.900%  395.26ms
 99.990%  395.26ms
 99.999%  395.26ms
100.000%  395.26ms
...
600 requests in 1.00m, 6.30MB read
Requests/sec:     10.00

When I run this: wrk -t5 -c10 -d1m -R10 --latency -s ./my_script.lua http://somewhere/something I get this:

  5 threads and 10 connections
  Thread calibration: mean lat.: 303.611ms, rate sampling interval: 995ms
  Thread calibration: mean lat.: 353.468ms, rate sampling interval: 960ms
  Thread calibration: mean lat.: 316.724ms, rate sampling interval: 1056ms
  Thread calibration: mean lat.: 347.817ms, rate sampling interval: 1013ms
  Thread calibration: mean lat.: 330.926ms, rate sampling interval: 994ms
...
 50.000%  359.93ms
 75.000%  453.12ms
 90.000%  539.65ms
 99.000%  619.01ms
 99.900%  673.79ms
 99.990%  673.79ms
 99.999%  673.79ms
100.000%  673.79ms
600 requests in 1.00m, 6.30MB read
Requests/sec:     10.00

Should number of connections

[IRus]

That mean that your server, possibly, bad at handling concurrent requests.

[dabos]

That's correct gmiejski 10 rps * 60 seconds = 600 total requests