fast-builder
fast-builder copied to clipboard
Published
20 hours ago •
takano-akio
takano-akio
Benchmarks and results?
In the package description on Hackage, you claim that:
In many benchmarks, the performance improvement is 2x-10x.
Which benchmarks are these? Are there some results we can look at?
You can find some benchmarks under the benchmarks directory: https://github.com/takano-akio/fast-builder/tree/master/benchmarks
On my machine they give results like this (bstr is the builder from the bytestring package, fast is this library):
Preprocessing benchmark 'aeson' for fast-builder-0.1.2.1..
Building benchmark 'aeson' for fast-builder-0.1.2.1..
Running 1 benchmarks...
Benchmark aeson: RUNNING...
benchmarking fast
time 640.8 μs (638.4 μs .. 642.8 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 644.0 μs (641.8 μs .. 647.3 μs)
std dev 9.285 μs (6.663 μs .. 14.98 μs)
benchmarking bstr
time 979.1 μs (974.4 μs .. 983.3 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 976.8 μs (972.9 μs .. 983.0 μs)
std dev 15.82 μs (11.40 μs .. 25.61 μs)
benchmarking aeson
time 951.6 μs (947.1 μs .. 956.4 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 955.0 μs (950.3 μs .. 959.6 μs)
std dev 15.23 μs (13.08 μs .. 17.89 μs)
Benchmark aeson: FINISH
Preprocessing benchmark 'map' for fast-builder-0.1.2.1..
Building benchmark 'map' for fast-builder-0.1.2.1..
Running 1 benchmarks...
Benchmark map: RUNNING...
benchmarking 10/lazy/fast
time 340.3 ns (335.0 ns .. 351.5 ns)
0.997 R² (0.991 R² .. 1.000 R²)
mean 337.5 ns (335.5 ns .. 344.5 ns)
std dev 11.80 ns (1.471 ns .. 24.91 ns)
variance introduced by outliers: 51% (severely inflated)
benchmarking 10/lazy/bstr
time 1.040 μs (1.035 μs .. 1.043 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.038 μs (1.035 μs .. 1.041 μs)
std dev 9.497 ns (7.459 ns .. 14.01 ns)
benchmarking 100/lazy/fast
time 2.562 μs (2.551 μs .. 2.574 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 2.562 μs (2.548 μs .. 2.590 μs)
std dev 68.43 ns (41.12 ns .. 118.4 ns)
variance introduced by outliers: 34% (moderately inflated)
benchmarking 100/lazy/bstr
time 7.249 μs (7.217 μs .. 7.284 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 7.258 μs (7.234 μs .. 7.291 μs)
std dev 88.56 ns (68.17 ns .. 114.4 ns)
benchmarking 1000/lazy/fast
time 23.21 μs (22.97 μs .. 23.47 μs)
0.999 R² (0.998 R² .. 1.000 R²)
mean 23.17 μs (23.00 μs .. 23.46 μs)
std dev 773.8 ns (490.4 ns .. 1.085 μs)
variance introduced by outliers: 37% (moderately inflated)
benchmarking 1000/lazy/bstr
time 73.13 μs (72.76 μs .. 73.52 μs)
1.000 R² (0.999 R² .. 1.000 R²)
mean 73.35 μs (72.85 μs .. 74.06 μs)
std dev 2.032 μs (1.466 μs .. 2.753 μs)
variance introduced by outliers: 26% (moderately inflated)
benchmarking 10000/lazy/fast
time 240.3 μs (237.2 μs .. 243.2 μs)
0.999 R² (0.999 R² .. 1.000 R²)
mean 237.6 μs (236.5 μs .. 239.4 μs)
std dev 4.541 μs (3.461 μs .. 5.861 μs)
variance introduced by outliers: 11% (moderately inflated)
benchmarking 10000/lazy/bstr
time 746.6 μs (740.0 μs .. 754.4 μs)
0.999 R² (0.999 R² .. 0.999 R²)
mean 752.5 μs (745.7 μs .. 757.8 μs)
std dev 20.54 μs (17.27 μs .. 24.64 μs)
variance introduced by outliers: 17% (moderately inflated)
Benchmark map: FINISH
Preprocessing benchmark 'vector' for fast-builder-0.1.2.1..
Building benchmark 'vector' for fast-builder-0.1.2.1..
Running 1 benchmarks...
Benchmark vector: RUNNING...
benchmarking 10/lazy/fast
time 240.0 ns (239.5 ns .. 240.7 ns)
1.000 R² (1.000 R² .. 1.000 R²)
mean 239.6 ns (239.3 ns .. 240.5 ns)
std dev 1.610 ns (687.3 ps .. 3.134 ns)
benchmarking 10/lazy/bstr
time 1.411 μs (1.410 μs .. 1.413 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.412 μs (1.411 μs .. 1.414 μs)
std dev 5.155 ns (3.017 ns .. 9.029 ns)
benchmarking 10/strict/fast
time 188.1 ns (187.6 ns .. 188.7 ns)
1.000 R² (0.998 R² .. 1.000 R²)
mean 188.8 ns (187.6 ns .. 194.5 ns)
std dev 7.543 ns (836.1 ps .. 17.25 ns)
variance introduced by outliers: 59% (severely inflated)
benchmarking 10/io/fast
time 406.7 ns (405.6 ns .. 408.0 ns)
1.000 R² (0.999 R² .. 1.000 R²)
mean 408.4 ns (405.7 ns .. 421.1 ns)
std dev 17.18 ns (1.748 ns .. 39.45 ns)
variance introduced by outliers: 60% (severely inflated)
benchmarking 10/io/bstr
time 1.291 μs (1.290 μs .. 1.293 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.295 μs (1.290 μs .. 1.327 μs)
std dev 34.56 ns (3.797 ns .. 84.23 ns)
variance introduced by outliers: 35% (moderately inflated)
benchmarking 100/lazy/fast
time 1.072 μs (1.067 μs .. 1.076 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.070 μs (1.068 μs .. 1.073 μs)
std dev 8.967 ns (7.523 ns .. 11.23 ns)
benchmarking 100/lazy/bstr
time 9.631 μs (9.621 μs .. 9.641 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 9.629 μs (9.623 μs .. 9.637 μs)
std dev 22.91 ns (17.17 ns .. 32.28 ns)
benchmarking 100/strict/fast
time 996.5 ns (995.1 ns .. 998.4 ns)
1.000 R² (1.000 R² .. 1.000 R²)
mean 996.2 ns (995.6 ns .. 998.0 ns)
std dev 3.345 ns (2.017 ns .. 5.791 ns)
benchmarking 100/io/fast
time 1.572 μs (1.569 μs .. 1.574 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.571 μs (1.570 μs .. 1.573 μs)
std dev 4.480 ns (3.588 ns .. 5.551 ns)
benchmarking 100/io/bstr
time 9.832 μs (9.825 μs .. 9.844 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 9.834 μs (9.830 μs .. 9.842 μs)
std dev 18.98 ns (10.99 ns .. 37.59 ns)
benchmarking 1000/lazy/fast
time 7.485 μs (7.482 μs .. 7.489 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 7.486 μs (7.483 μs .. 7.491 μs)
std dev 12.69 ns (8.605 ns .. 19.19 ns)
benchmarking 1000/lazy/bstr
time 91.89 μs (91.84 μs .. 91.95 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 91.78 μs (91.72 μs .. 91.94 μs)
std dev 300.1 ns (142.5 ns .. 604.3 ns)
benchmarking 1000/strict/fast
time 7.462 μs (7.456 μs .. 7.472 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 7.459 μs (7.456 μs .. 7.464 μs)
std dev 12.27 ns (8.676 ns .. 19.23 ns)
benchmarking 1000/io/fast
time 8.353 μs (8.341 μs .. 8.361 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 8.340 μs (8.332 μs .. 8.351 μs)
std dev 31.41 ns (23.18 ns .. 47.08 ns)
benchmarking 1000/io/bstr
time 116.7 μs (115.9 μs .. 117.6 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 117.0 μs (116.6 μs .. 117.5 μs)
std dev 1.419 μs (1.244 μs .. 1.591 μs)
benchmarking 10000/lazy/fast
time 66.72 μs (66.62 μs .. 66.80 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 66.62 μs (66.56 μs .. 66.69 μs)
std dev 221.7 ns (183.3 ns .. 282.3 ns)
benchmarking 10000/lazy/bstr
time 953.1 μs (951.4 μs .. 954.5 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 951.4 μs (950.0 μs .. 952.7 μs)
std dev 4.620 μs (3.387 μs .. 6.447 μs)
benchmarking 10000/strict/fast
time 66.28 μs (66.26 μs .. 66.30 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 66.28 μs (66.27 μs .. 66.30 μs)
std dev 52.49 ns (45.15 ns .. 65.26 ns)
benchmarking 10000/io/fast
time 73.20 μs (73.13 μs .. 73.27 μs)
1.000 R² (1.000 R² .. 1.000 R²)
mean 73.16 μs (73.14 μs .. 73.19 μs)
std dev 77.65 ns (54.62 ns .. 118.6 ns)
benchmarking 10000/io/bstr
time 1.394 ms (1.391 ms .. 1.398 ms)
1.000 R² (1.000 R² .. 1.000 R²)
mean 1.395 ms (1.393 ms .. 1.398 ms)
std dev 9.151 μs (7.170 μs .. 14.18 μs)
Benchmark vector: FINISH
Thank you for providing this! It would be good to add this to the README (along with the results).