wasmi icon indicating copy to clipboard operation
wasmi copied to clipboard

Refactor `wasmi` bytecode

Open Robbepop opened this issue 2 years ago • 2 comments

Robbepop avatar Oct 15 '22 08:10 Robbepop

BENCHMARKS

NATIVEWASMTIME
BENCHMARKMASTERPRDIFFMASTERPRDIFFWASMTIME OVERHEAD
execute/
bare_call_0
1.23ms 1.20ms :green_circle: -1.98% 902.71µs 908.39µs :red_circle: 0.65% :green_circle: -25%
execute/
bare_call_0/typed
652.44µs 651.51µs :white_circle: 0.02% 389.28µs 408.21µs :red_circle: 4.85% :green_circle: -37%
execute/
bare_call_1
1.33ms 1.32ms :green_circle: -0.73% 1.11ms 1.22ms :red_circle: 10.07% :green_circle: -8%
execute/
bare_call_16
3.07ms 2.85ms :green_circle: -7.35% 4.60ms 4.64ms :white_circle: 0.07% :yellow_circle: 63%
execute/
bare_call_16/typed
1.53ms 1.36ms :green_circle: -11.26% 2.00ms 2.02ms :white_circle: 0.68% :green_circle: 48%
execute/
bare_call_1/typed
776.18µs 764.30µs :green_circle: -1.43% 699.57µs 697.45µs :white_circle: -0.45% :green_circle: -9%
execute/
bare_call_4
1.65ms 1.64ms :green_circle: -0.70% 1.72ms 1.75ms :red_circle: 1.56% :green_circle: 7%
execute/
bare_call_4/typed
876.78µs 857.93µs :green_circle: -2.09% 824.43µs 802.98µs :green_circle: -2.70% :green_circle: -6%
execute/
br_table
830.81µs 890.46µs :white_circle: 5.23% 878.53µs 920.41µs :red_circle: 4.49% :green_circle: 3%
execute/
count_until
1.15ms 772.43µs :green_circle: -33.07% 1.98ms 2.37ms :red_circle: 19.67% :red_circle: 207%
execute/
factorial_iterative
500.72µs 364.59µs :green_circle: -27.17% 801.11µs 842.85µs :red_circle: 5.38% :red_circle: 131%
execute/
factorial_recursive
761.11µs 700.12µs :green_circle: -8.19% 1.23ms 1.27ms :red_circle: 3.64% :yellow_circle: 82%
execute/
fib_iterative
2.54ms 1.67ms :green_circle: -34.17% 4.33ms 4.48ms :red_circle: 3.69% :red_circle: 168%
execute/
fib_recursive
7.98ms 6.64ms :green_circle: -16.87% 11.19ms 11.29ms :white_circle: 0.86% :yellow_circle: 70%
execute/
global_bump
2.00ms 1.17ms :green_circle: -41.59% 3.00ms 3.17ms :red_circle: 5.57% :red_circle: 171%
execute/
global_const
1.21ms 898.46µs :green_circle: -26.18% 2.21ms 2.33ms :red_circle: 5.80% :red_circle: 159%
execute/
host_calls
32.42µs 31.52µs :green_circle: -2.72% 35.79µs 37.24µs :red_circle: 4.00% :green_circle: 18%
execute/
memory_fill
2.04ms 1.50ms :green_circle: -26.73% 3.74ms 3.99ms :red_circle: 6.75% :red_circle: 166%
execute/
memory_sum
2.02ms 1.50ms :green_circle: -25.87% 3.80ms 3.97ms :red_circle: 4.58% :red_circle: 165%
execute/
memory_vec_add
4.42ms 3.16ms :green_circle: -28.56% 7.72ms 8.22ms :red_circle: 6.48% :red_circle: 161%
execute/
recursive_is_even
1.33ms 1.28ms :green_circle: -4.21% 2.02ms 2.10ms :red_circle: 4.08% :yellow_circle: 64%
execute/
recursive_ok
171.82µs 165.80µs :green_circle: -3.33% 282.62µs 295.25µs :red_circle: 4.46% :yellow_circle: 78%
execute/
recursive_scan
219.00µs 200.94µs :green_circle: -8.34% 359.73µs 375.06µs :red_circle: 4.10% :yellow_circle: 87%
execute/
recursive_trap
16.95µs 16.19µs :green_circle: -4.29% 28.84µs 28.87µs :white_circle: 0.13% :yellow_circle: 78%
execute/
regex_redux
815.50µs 660.73µs :green_circle: -18.75% 1.40ms 1.45ms :red_circle: 3.17% :red_circle: 119%
execute/
rev_complement
890.52µs 620.62µs :green_circle: -30.32% 1.39ms 1.43ms :red_circle: 2.87% :red_circle: 130%
execute/
tiny_keccak
784.55µs 470.76µs :green_circle: -39.94% 1.09ms 1.19ms :red_circle: 9.96% :red_circle: 153%
execute/
trunc_f2i
1.33ms 1.09ms :green_circle: -17.77% 2.22ms 2.44ms :red_circle: 9.77% :red_circle: 123%
instantiate/
wasm_kernel
70.63µs 70.61µs :white_circle: 1.16% 71.43µs 95.52µs :red_circle: 32.71% :green_circle: 35%
translate/
erc1155
239.75µs 246.43µs :red_circle: 2.37% 373.63µs 380.49µs :red_circle: 1.99% :yellow_circle: 54%
translate/
erc20
117.87µs 119.29µs :white_circle: 1.45% 181.39µs 185.79µs :red_circle: 2.23% :yellow_circle: 56%
translate/
erc721
167.93µs 169.41µs :red_circle: 1.44% 264.59µs 270.11µs :red_circle: 1.97% :yellow_circle: 59%
translate/
spidermonkey
0.00ns 0.00ns :white_circle: 0.53% 0.00ns 0.00ns :red_circle: 1.99% :green_circle: 0%
translate/
wasm_kernel
4.26ms 4.19ms :green_circle: -1.92% 7.07ms 7.19ms :red_circle: 2.16% :yellow_circle: 72%

Link to pipeline

paritytech-cicd-pr avatar Oct 15 '22 08:10 paritytech-cicd-pr

I used the cargo-asm tool to inspect the Executor::execute function on an assembly level to understand if Rust and LLVM somehow fail to properly optimize the function after removal of the 4 unnecessary instructions. The result is: There is a huge difference in the assembly output!

See yourself:

  • master: https://gist.github.com/Robbepop/cde5a25f00b78259a11170a6614aca90
    • Roughly 3.8k lines of assembly
  • PR: https://gist.github.com/Robbepop/9df0eb661c5fcab9aa8391221fda7196
    • Roughly 5.9k lines of assembly

Robbepop avatar Oct 15 '22 09:10 Robbepop

Codecov Report

Merging #518 (4b3d74d) into master (f5e1c46) will decrease coverage by 0.07%. The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #518      +/-   ##
==========================================
- Coverage   79.73%   79.65%   -0.08%     
==========================================
  Files          75       75              
  Lines        6291     6267      -24     
==========================================
- Hits         5016     4992      -24     
  Misses       1275     1275              
Impacted Files Coverage Δ
crates/wasmi/src/engine/bytecode/mod.rs 100.00% <ø> (ø)
crates/wasmi/src/engine/executor.rs 98.04% <ø> (-0.05%) :arrow_down:
crates/wasmi/src/engine/bytecode/tests.rs 100.00% <100.00%> (ø)
crates/wasmi/src/engine/func_builder/mod.rs 89.72% <100.00%> (-0.13%) :arrow_down:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

codecov-commenter avatar Oct 23 '22 10:10 codecov-commenter

While the changes introduces in this PR were a regression to performance for wasmi with Rust 1.64 they are now significantly improving wasmi performance with Rust 1.65. Therefore it is now ready to merge.

Robbepop avatar Nov 04 '22 10:11 Robbepop