wasmi
wasmi copied to clipboard
wasmi-labs
Add differential execution fuzz testing
This will fuzz test executions of generated Wasm module functions with new wasmi versions against the battle-tested legacy wasmi version 0.13.0.
Codecov Report
Merging #450 (41db837) into master (52c8a0a) will decrease coverage by
0.62%. The diff coverage is0.00%.
@@ Coverage Diff @@
## master #450 +/- ##
==========================================
- Coverage 79.71% 79.08% -0.63%
==========================================
Files 75 76 +1
Lines 6284 6334 +50
==========================================
Hits 5009 5009
- Misses 1275 1325 +50
| Impacted Files | Coverage Δ | |
|---|---|---|
| fuzz/fuzz_targets/differential.rs | 0.00% <0.00%> (ø) |
:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more
BENCHMARKS
| NATIVE | WASMTIME | ||||||
|---|---|---|---|---|---|---|---|
| BENCHMARK | MASTER | PR | DIFF | MASTER | PR | DIFF | WASMTIME OVERHEAD |
| execute/ bare_call_0 | 1.28ms | 1.27ms | :white_circle: -2.43% | 1.10ms | 1.10ms | :white_circle: -0.17% | :green_circle: -13% |
| execute/ bare_call_0/typed | 699.74µs | 698.36µs | :white_circle: -0.29% | 543.08µs | 546.59µs | :white_circle: 0.88% | :green_circle: -22% |
| execute/ bare_call_1 | 1.40ms | 1.40ms | :white_circle: -0.06% | 1.35ms | 1.35ms | :red_circle: -0.01% | :green_circle: -4% |
| execute/ bare_call_16 | 2.92ms | 2.90ms | :white_circle: -0.60% | 5.08ms | 5.07ms | :red_circle: -0.05% | :yellow_circle: 75% |
| execute/ bare_call_16/typed | 1.30ms | 1.28ms | :white_circle: -1.56% | 2.33ms | 2.39ms | :red_circle: 2.90% | :yellow_circle: 86% |
| execute/ bare_call_1/typed | 800.33µs | 800.42µs | :white_circle: -0.02% | 868.18µs | 865.78µs | :white_circle: -0.19% | :green_circle: 8% |
| execute/ bare_call_4 | 1.74ms | 1.79ms | :red_circle: 2.60% | 2.15ms | 2.15ms | :white_circle: 0.05% | :green_circle: 21% |
| execute/ bare_call_4/typed | 889.15µs | 889.67µs | :white_circle: 0.07% | 1.00ms | 999.30µs | :white_circle: -0.35% | :green_circle: 12% |
| execute/ br_table | 909.60µs | 950.52µs | :red_circle: 4.47% | 1.12ms | 1.13ms | :white_circle: 0.38% | :green_circle: 19% |
| execute/ count_until | 902.46µs | 903.18µs | :white_circle: 0.11% | 2.35ms | 2.35ms | :white_circle: 0.05% | :red_circle: 160% |
| execute/ factorial_iterative | 364.52µs | 363.50µs | :white_circle: -0.19% | 936.93µs | 946.27µs | :red_circle: 1.07% | :red_circle: 160% |
| execute/ factorial_recursive | 739.53µs | 755.59µs | :red_circle: 2.16% | 1.53ms | 1.54ms | :white_circle: 0.89% | :red_circle: 104% |
| execute/ fib_iterative | 1.79ms | 1.79ms | :white_circle: 0.04% | 4.97ms | 5.08ms | :red_circle: 2.27% | :red_circle: 184% |
| execute/ fib_recursive | 7.81ms | 6.86ms | :green_circle: -11.95% | 14.17ms | 14.14ms | :white_circle: -0.13% | :red_circle: 106% |
| execute/ global_bump | 1.36ms | 1.36ms | :white_circle: -0.15% | 3.81ms | 3.81ms | :white_circle: -0.01% | :red_circle: 180% |
| execute/ global_const | 1.00ms | 1.00ms | :white_circle: 0.13% | 2.76ms | 2.77ms | :white_circle: 0.13% | :red_circle: 177% |
| execute/ host_calls | 32.69µs | 32.66µs | :white_circle: -0.08% | 44.54µs | 44.57µs | :white_circle: 0.04% | :green_circle: 36% |
| execute/ memory_fill | 1.56ms | 1.56ms | :white_circle: -0.02% | 4.17ms | 4.17ms | :white_circle: 0.15% | :red_circle: 168% |
| execute/ memory_sum | 1.55ms | 1.55ms | :white_circle: -0.04% | 4.15ms | 4.33ms | :red_circle: 4.49% | :red_circle: 179% |
| execute/ memory_vec_add | 3.05ms | 3.06ms | :white_circle: -0.02% | 8.86ms | 8.85ms | :white_circle: 0.07% | :red_circle: 190% |
| execute/ recursive_is_even | 1.32ms | 1.30ms | :white_circle: -1.69% | 2.60ms | 2.68ms | :red_circle: 2.96% | :red_circle: 106% |
| execute/ recursive_ok | 170.75µs | 172.85µs | :red_circle: 1.23% | 359.05µs | 359.63µs | :white_circle: 0.19% | :red_circle: 108% |
| execute/ recursive_scan | 213.43µs | 214.32µs | :white_circle: 0.25% | 466.96µs | 466.11µs | :white_circle: -0.06% | :red_circle: 117% |
| execute/ recursive_trap | 17.31µs | 16.91µs | :white_circle: -1.32% | 36.22µs | 36.29µs | :white_circle: 0.14% | :red_circle: 115% |
| execute/ regex_redux | 666.26µs | 662.67µs | :white_circle: -1.07% | 1.60ms | 1.59ms | :white_circle: -0.89% | :red_circle: 140% |
| execute/ rev_complement | 596.65µs | 592.28µs | :white_circle: -0.37% | 1.55ms | 1.55ms | :white_circle: -0.08% | :red_circle: 162% |
| execute/ tiny_keccak | 482.84µs | 467.90µs | :green_circle: -2.91% | 1.38ms | 1.37ms | :white_circle: -0.72% | :red_circle: 192% |
| execute/ trunc_f2i | 1.02ms | 1.02ms | :white_circle: -0.05% | 2.55ms | 2.55ms | :white_circle: -0.17% | :red_circle: 150% |
| instantiate/ wasm_kernel | 72.11µs | 69.19µs | :white_circle: -2.35% | 124.36µs | 124.30µs | :white_circle: 0.05% | :yellow_circle: 80% |
| translate/ wasm_kernel | 4.21ms | 4.18ms | :white_circle: -0.33% | 7.76ms | 7.76ms | :white_circle: -0.34% | :yellow_circle: 85% |
