k6
k6 copied to clipboard
Published
20 hours ago •
grafana
grafana
cloud/output: Atlas friendly benchmarks
This is the current situation between master and atlas branches. I updated the benchmark for not generating a new Registry for each new bench iteration.
~~Apparently the easyjson part is not so happy, I haven't found an obvious reason, I need to isolate the EasyJSON function in a dedicated benchmark.~~
name old time/op new time/op delta
AggregateHTTP/tags:1-4 185ms ± 6% 37ms ±11% -79.90% (p=0.008 n=5+5)
AggregateHTTP/tags:5-4 218ms ± 6% 53ms ±11% -75.45% (p=0.008 n=5+5)
AggregateHTTP/tags:35-4 225ms ± 3% 60ms ±10% -73.42% (p=0.008 n=5+5)
AggregateHTTP/tags:315-4 310ms ± 3% 61ms ±12% -80.31% (p=0.008 n=5+5)
AggregateHTTP/tags:3645-4 532ms ± 3% 60ms ±12% -88.79% (p=0.008 n=5+5)
name old time/op new time/op delta
MetricMarshal/10000-4 36.5ms ± 7% 25.7ms ±10% -29.55% (p=0.008 n=5+5)
MetricMarshal/100000-4 168ms ± 7% 136ms ±10% -19.18% (p=0.008 n=5+5)
MetricMarshal/500000-4 736ms ± 1% 595ms ± 4% -19.13% (p=0.016 n=4+5)
MetricMarshalWriter/10000-4 34.1ms ±15% 26.2ms ± 4% -23.07% (p=0.008 n=5+5)
MetricMarshalWriter/100000-4 170ms ± 7% 144ms ± 3% -14.97% (p=0.008 n=5+5)
MetricMarshalWriter/500000-4 733ms ± 2% 599ms ± 4% -18.23% (p=0.008 n=5+5)
MetricMarshalGzip/10000_bestcompression-4 134ms ± 3% 132ms ± 4% ~ (p=0.548 n=5+5)
MetricMarshalGzip/10000_default-4 57.6ms ± 5% 52.7ms ± 2% -8.51% (p=0.008 n=5+5)
MetricMarshalGzip/10000_bestspeed-4 20.9ms ± 4% 19.7ms ±15% ~ (p=0.310 n=5+5)
MetricMarshalGzip/100000_bestcompression-4 1.22s ± 3% 1.26s ± 3% ~ (p=0.056 n=5+5)
MetricMarshalGzip/100000_default-4 519ms ± 1% 475ms ± 2% -8.51% (p=0.008 n=5+5)
MetricMarshalGzip/100000_bestspeed-4 140ms ± 3% 138ms ± 5% ~ (p=0.310 n=5+5)
MetricMarshalGzip/500000_default-4 2.62s ± 1% 2.39s ± 1% -8.45% (p=0.008 n=5+5)
MetricMarshalGzip/500000_bestspeed-4 697ms ± 2% 676ms ± 3% ~ (p=0.095 n=5+5)
MetricMarshalGzip/500000_bestcompression-4 6.14s ± 1% 6.14s ± 3% ~ (p=0.421 n=5+5)
MetricMarshalGzipAll/10000_bestspeed-4 67.1ms ± 9% 58.4ms ± 2% -12.99% (p=0.008 n=5+5)
MetricMarshalGzipAll/100000_bestspeed-4 340ms ±13% 303ms ± 6% -10.98% (p=0.016 n=5+5)
MetricMarshalGzipAll/500000_bestspeed-4 1.51s ± 3% 1.34s ± 3% -11.79% (p=0.008 n=5+5)
MetricMarshalGzipAllWriter/10000_bestspeed-4 63.5ms ± 4% 54.5ms ± 6% -14.11% (p=0.008 n=5+5)
MetricMarshalGzipAllWriter/100000_bestspeed-4 332ms ± 6% 286ms ± 4% -13.89% (p=0.008 n=5+5)
MetricMarshalGzipAllWriter/500000_bestspeed-4 1.49s ± 3% 1.30s ± 8% -12.96% (p=0.008 n=5+5)
name old speed new speed delta
MetricMarshal/10000-4 114MB/s ± 7% 161MB/s ± 9% +42.12% (p=0.008 n=5+5)
MetricMarshal/100000-4 247MB/s ± 7% 306MB/s ± 9% +23.86% (p=0.008 n=5+5)
MetricMarshal/500000-4 281MB/s ± 1% 348MB/s ± 4% +23.71% (p=0.016 n=4+5)
MetricMarshalWriter/10000-4 122MB/s ±16% 158MB/s ± 4% +29.23% (p=0.008 n=5+5)
MetricMarshalWriter/100000-4 245MB/s ± 8% 287MB/s ± 3% +17.35% (p=0.008 n=5+5)
MetricMarshalWriter/500000-4 282MB/s ± 2% 345MB/s ± 4% +22.33% (p=0.008 n=5+5)
MetricMarshalGzip/10000_bestcompression-4 31.0MB/s ± 3% 31.4MB/s ± 4% ~ (p=0.548 n=5+5)
MetricMarshalGzip/10000_default-4 71.8MB/s ± 5% 78.4MB/s ± 2% +9.23% (p=0.008 n=5+5)
MetricMarshalGzip/10000_bestspeed-4 197MB/s ± 4% 212MB/s ±16% ~ (p=0.310 n=5+5)
MetricMarshalGzip/100000_bestcompression-4 33.8MB/s ± 3% 32.8MB/s ± 3% ~ (p=0.056 n=5+5)
MetricMarshalGzip/100000_default-4 79.7MB/s ± 1% 87.1MB/s ± 2% +9.31% (p=0.008 n=5+5)
MetricMarshalGzip/100000_bestspeed-4 295MB/s ± 3% 301MB/s ± 5% ~ (p=0.310 n=5+5)
MetricMarshalGzip/500000_default-4 79.1MB/s ± 1% 86.4MB/s ± 1% +9.23% (p=0.008 n=5+5)
MetricMarshalGzip/500000_bestspeed-4 297MB/s ± 2% 306MB/s ± 3% ~ (p=0.095 n=5+5)
MetricMarshalGzip/500000_bestcompression-4 33.7MB/s ± 1% 33.7MB/s ± 3% ~ (p=0.341 n=5+5)
MetricMarshalGzipAll/10000_bestspeed-4 61.8MB/s ± 8% 70.8MB/s ± 2% +14.64% (p=0.008 n=5+5)
MetricMarshalGzipAll/100000_bestspeed-4 122MB/s ±12% 137MB/s ± 6% +12.06% (p=0.016 n=5+5)
MetricMarshalGzipAll/500000_bestspeed-4 137MB/s ± 3% 155MB/s ± 3% +13.37% (p=0.008 n=5+5)
MetricMarshalGzipAllWriter/10000_bestspeed-4 65.1MB/s ± 4% 75.9MB/s ± 6% +16.52% (p=0.008 n=5+5)
MetricMarshalGzipAllWriter/100000_bestspeed-4 125MB/s ± 6% 145MB/s ± 4% +16.04% (p=0.008 n=5+5)
MetricMarshalGzipAllWriter/500000_bestspeed-4 139MB/s ± 3% 160MB/s ± 7% +15.05% (p=0.008 n=5+5)
name old ratio new ratio delta
MetricMarshalGzip/10000_bestcompression-4 15.5 ± 0% 15.8 ± 0% +1.78% (p=0.008 n=5+5)
MetricMarshalGzip/10000_default-4 13.8 ± 0% 14.2 ± 0% +2.56% (p=0.008 n=5+5)
MetricMarshalGzip/10000_bestspeed-4 6.80 ± 0% 7.19 ± 0% +5.77% (p=0.008 n=5+5)
MetricMarshalGzip/100000_bestcompression-4 14.3 ± 0% 15.5 ± 1% +8.10% (p=0.016 n=4+5)
MetricMarshalGzip/100000_default-4 13.2 ± 0% 14.2 ± 0% +7.11% (p=0.008 n=5+5)
MetricMarshalGzip/100000_bestspeed-4 6.80 ± 0% 7.20 ± 0% +5.82% (p=0.008 n=5+5)
MetricMarshalGzip/500000_default-4 13.2 ± 0% 14.1 ± 0% +6.72% (p=0.008 n=5+5)
MetricMarshalGzip/500000_bestspeed-4 6.80 ± 0% 7.20 ± 0% +5.83% (p=0.008 n=5+5)
MetricMarshalGzip/500000_bestcompression-4 14.3 ± 0% 15.4 ± 0% +8.05% (p=0.008 n=5+5)
name old time/op new time/op delta
HTTPPush/count:1000-4 9.03ms ± 1% 7.87ms ± 1% -12.88% (p=0.008 n=5+5)
HTTPPush/count:5000-4 30.3ms ±12% 27.4ms ±11% ~ (p=0.151 n=5+5)
HTTPPush/count:50000-4 156ms ± 7% 149ms ± 2% ~ (p=0.222 n=5+5)
HTTPPush/count:100000-4 304ms ± 3% 282ms ± 5% -7.44% (p=0.008 n=5+5)
HTTPPush/count:250000-4 766ms ± 4% 663ms ± 5% -13.38% (p=0.008 n=5+5)
name old time/op new time/op delta
NewSampleFromTrail/no_failed-4 803ns ±10% 1781ns ± 8% +121.84% (p=0.000 n=10+10)
NewSampleFromTrail/failed-4 1.15µs ± 3% 2.23µs ± 2% +94.39% (p=0.000 n=9+9)
The increment of NewSampleFromTrail sounds fine because we added the following new lines in the atlas branch:
https://github.com/grafana/k6/blob/be8ca6fe61c5961d83399a76f49d5aee07e94a91/output/cloud/data.go#L114-L118