hop icon indicating copy to clipboard operation
hop copied to clipboard

front page results about "25x faster" are incorrect

Open theamk opened this issue 3 years ago • 1 comments

The tests on the front page have extremely small total times, a single milliseconds range. This means you are measuring mostly startup overhead and not actually decompression (the benchmarking harness tells you the same thing as well: "Command took less than 5 milliseconds, results may be inaccurate")

In order for performance tests to be accurate, you need to re-measure it with larger archives and more data, so that overall time is no longer dominated by program startup.

theamk avatar Nov 11 '21 04:11 theamk

You're probably right I should show with a bigger file, but 3.8 MB is the upper end of what I expect common real-world usage to look like. JavaScript source files tend not to be much larger than that.

Here's the same file on an Ubuntu Linux AMD64 machine. Note the --warmup=2000

image

Jarred-Sumner avatar Nov 11 '21 05:11 Jarred-Sumner