wasmi
wasmi copied to clipboard
wasmi-labs
Compare benchmarks between `rustc` stable and nightly channels
I got suspicious after I saw huge regressions in the ballpark of 30-80% on my local development platform so I just wanted to make sure that it is not my local environment by running those same benchmarks on the CI runners.
edit: Those regressions are very real ...
BENCHMARKS
| NATIVE | WASMTIME | ||||||
|---|---|---|---|---|---|---|---|
| BENCHMARK | MASTER | PR | DIFF | MASTER | PR | DIFF | WASMTIME OVERHEAD |
| execute/ count_until | 888.50µs | 1.24ms | :red_circle: 40.06% | 2.38ms | 3.10ms | :red_circle: 30.07% | :red_circle: 149% |
| execute/ factorial_iterative | 400.45ns | 542.71ns | :red_circle: 35.46% | 993.53ns | 1.30µs | :red_circle: 30.67% | :red_circle: 139% |
| execute/ factorial_recursive | 761.07ns | 840.76ns | :red_circle: 10.47% | 1.56µs | 1.71µs | :red_circle: 9.65% | :red_circle: 103% |
| execute/ fib_iterative | 1.94ms | 2.55ms | :red_circle: 36.62% | 5.12ms | 6.72ms | :red_circle: 31.43% | :red_circle: 163% |
| execute/ fib_recursive | 6.88ms | 8.46ms | :red_circle: 23.79% | 14.30ms | 15.56ms | :red_circle: 9.01% | :yellow_circle: 84% |
| execute/ global_bump | 1.43ms | 1.83ms | :red_circle: 27.29% | 3.63ms | 4.53ms | :red_circle: 24.79% | :red_circle: 148% |
| execute/ host_calls | 31.80µs | 32.44µs | :white_circle: 2.03% | 45.05µs | 48.33µs | :red_circle: 7.37% | :green_circle: 49% |
| execute/ memory_fill | 1.54ms | 2.32ms | :red_circle: 50.88% | 4.33ms | 5.81ms | :red_circle: 34.03% | :red_circle: 151% |
| execute/ memory_sum | 1.52ms | 2.34ms | :red_circle: 53.50% | 4.37ms | 5.84ms | :red_circle: 33.68% | :red_circle: 150% |
| execute/ memory_vec_add | 3.03ms | 4.81ms | :red_circle: 60.16% | 8.97ms | 12.11ms | :red_circle: 35.19% | :red_circle: 152% |
| execute/ recursive_is_even | 1.30ms | 1.35ms | :red_circle: 3.91% | 2.60ms | 2.69ms | :red_circle: 3.15% | :yellow_circle: 99% |
| execute/ recursive_ok | 173.10µs | 181.42µs | :red_circle: 5.00% | 350.49µs | 380.08µs | :red_circle: 8.82% | :red_circle: 109% |
| execute/ recursive_scan | 213.44µs | 235.58µs | :red_circle: 10.47% | 468.96µs | 505.36µs | :red_circle: 7.65% | :red_circle: 115% |
| execute/ recursive_trap | 17.29µs | 17.99µs | :red_circle: 4.15% | 35.50µs | 39.14µs | :red_circle: 10.18% | :red_circle: 118% |
| execute/ regex_redux | 665.58µs | 871.56µs | :red_circle: 30.94% | 1.60ms | 2.08ms | :red_circle: 29.51% | :red_circle: 138% |
| execute/ rev_complement | 596.99µs | 876.68µs | :red_circle: 46.98% | 1.55ms | 2.11ms | :red_circle: 35.85% | :red_circle: 140% |
| execute/ tiny_keccak | 473.22µs | 833.78µs | :red_circle: 77.02% | 1.42ms | 1.80ms | :red_circle: 26.52% | :red_circle: 116% |
| execute/ trunc_f2i | 1.07ms | 1.31ms | :red_circle: 22.19% | 2.73ms | 3.49ms | :red_circle: 27.80% | :red_circle: 167% |
| instantiate/ wasm_kernel | 70.04µs | 71.45µs | :white_circle: 2.09% | 128.03µs | 92.10µs | :green_circle: -27.74% | :green_circle: 29% |
| translate/ wasm_kernel | 4.88ms | 4.72ms | :green_circle: -3.24% | 9.47ms | 9.88ms | :red_circle: 3.72% | :red_circle: 109% |
Closing this since the demonstration for https://github.com/rust-lang/rust/issues/102952 has been shown.