website icon indicating copy to clipboard operation
website copied to clipboard

Wrong Latency unit

Open georgmu opened this issue 2 years ago • 1 comments

Looking at the results of the web framework benchmark, I was very suspicious regarding the latencies.

The website states these are milliseconds (ms), but they looked more like microseconds (us or µs).

Looking at the wrk homepage confirmed it is microseconds, not milliseconds: https://github.com/wg/wrk/blob/4.2.0/SCRIPTING#L96

So either the latency unit should be changed to us or µs or the value should by divided by 1000.

georgmu avatar Aug 05 '22 11:08 georgmu

You are right average latency for starlette should be about 1.26ms instead of https://web-frameworks-benchmark.netlify.app/result?asc=0&f=starlette&metric=averageLatency&order_by=level64

Running 15s test @ http://172.17.0.2:3000/" +
  8 threads and 64 connections\n" +
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     1.26ms  758.59us  16.29ms   81.20%
    Req/Sec     6.61k     1.69k   15.97k    77.17%
  790328 requests in 15.07s, 100.24MB read

Thanks for your deep check on this

cc @SuspiciousLookingOwl

waghanza avatar Aug 23 '22 20:08 waghanza

This issue seems to be fixed only on the /result page, but not on the /compare page. For example, Ditsmod has a latency of 1ms with "concurrency 64", but the /compare page shows a thousand times more.

KostyaTretyak avatar Sep 25 '23 21:09 KostyaTretyak