WebAssembly
Dart deferred loading meta issue
The Dart team is experimenting with using binaryen tools to support our deferred loading feature for users targeting wasm. We currently have experimental support for the feature by having the compiler emit multiple modules.
Typically, when emitting a single module, we use wasm-opt with the --closed-world flag to optimize the wasm module that dart2wasm is emitting. This works well for us and sometimes produces 3x code size improvements as well as other structural improvements like inlining, devirtualization, DCE, etc.
Now with deferred loading we are now splitting that single module into many modules that collectively represent the same program. So we technically still have a closed world... but split across multiple modules. Ideally we could specify all these modules to wasm-opt and it would perform the same (or close to the same) optimizations collectively across all these modules. Today wasm-opt doesn't support anything like this.
I've tried implementing this using a combination of wasm-merge -> wasm-opt -> wasm-split but ran into several issues where the code didn't load and the code size didn't improve much.
Associated issues: https://github.com/WebAssembly/binaryen/issues/7725 https://github.com/WebAssembly/binaryen/issues/7724 https://github.com/WebAssembly/binaryen/issues/7723 https://github.com/WebAssembly/binaryen/issues/7722 https://github.com/WebAssembly/binaryen/issues/7721
Are you seeing that the full type section is duplicated across all of your split modules after wasm-split? If so, you can try running --minimize-recursion-groups as the last optimization in the wasm-opt stage. That could significantly improve the final cumulative code size.
cc @aheejin
Our invocation of wasm-opt was already passing --minimize-recursion-groups so I think the type section should be okay. I think most of the bloat comes from the export section.
I've wanted to analyze the size of the different sections in the split file. Normally I would do that using wami --section-stats <my_file.wasm>. But that tool complains there are too many exports in the main module and it gives up. Do you know of any other tools that can help me analyze the wasm file (or workarounds to that limit)?
https://github.com/google/bloaty has support for wasm files, and shows section sizes.
How many exports are there, btw? (can list them with wasm-opt --metrics -all)
wasm-opt --metrics -all works:
The original main module file, before the merge->opt->split pass, has 18041 exports.
After the passes are done, the main module has 165168 exports.
I got bloaty running too and see the following.
Before:
FILE SIZE VM SIZE
-------------- --------------
52.0% 6.48Mi NAN% 0 name
19.4% 2.41Mi NAN% 0 Code
12.9% 1.60Mi NAN% 0 Global
8.0% 1014Ki NAN% 0 Import
2.7% 338Ki NAN% 0 Type
2.0% 251Ki NAN% 0 Export
1.3% 169Ki NAN% 0 Element
1.0% 126Ki NAN% 0 Data
0.6% 82.2Ki NAN% 0 Function
0.1% 17.1Ki NAN% 0 Table
0.0% 50 NAN% 0 sourceMappingURL
0.0% 8 NAN% 0 [WASM Header]
0.0% 6 NAN% 0 Event
0.0% 4 NAN% 0 Start
0.0% 3 NAN% 0 DataCount
100.0% 12.5Mi 100.0% 0 TOTAL
After:
FILE SIZE VM SIZE
-------------- --------------
40.6% 7.60Mi NAN% 0 name
24.2% 4.53Mi NAN% 0 Import
13.6% 2.54Mi NAN% 0 Code
9.8% 1.84Mi NAN% 0 Global
7.2% 1.35Mi NAN% 0 Export
2.0% 386Ki NAN% 0 Element
1.5% 295Ki NAN% 0 Type
0.7% 126Ki NAN% 0 Data
0.4% 76.6Ki NAN% 0 Function
0.1% 16.2Ki NAN% 0 Table
0.0% 50 NAN% 0 sourceMappingURL
0.0% 8 NAN% 0 [WASM Header]
0.0% 6 NAN% 0 Event
0.0% 4 NAN% 0 Start
0.0% 3 NAN% 0 DataCount
100.0% 18.7Mi 100.0% 0 TOTAL
FWIW here are the sizes of the intermediate files.
After merging 350 modules:
FILE SIZE VM SIZE
-------------- --------------
51.5% 20.7Mi NAN% 0 name
26.6% 10.7Mi NAN% 0 Code
11.3% 4.54Mi NAN% 0 Import
5.5% 2.21Mi NAN% 0 Global
2.5% 1.00Mi NAN% 0 Type
0.9% 365Ki NAN% 0 Function
0.8% 335Ki NAN% 0 Element
0.6% 252Ki NAN% 0 Export
0.3% 126Ki NAN% 0 Data
0.0% 16.6Ki NAN% 0 Table
0.0% 50 NAN% 0 sourceMappingURL
0.0% 8 NAN% 0 [WASM Header]
0.0% 7 NAN% 0 Event
0.0% 4 NAN% 0 Start
0.0% 3 NAN% 0 DataCount
100.0% 40.2Mi 100.0% 0 TOTAL
After optimizing merged module:
FILE SIZE VM SIZE
-------------- --------------
39.3% 8.97Mi NAN% 0 name
27.6% 6.30Mi NAN% 0 Code
19.8% 4.53Mi NAN% 0 Import
8.1% 1.86Mi NAN% 0 Global
1.6% 369Ki NAN% 0 Type
1.4% 329Ki NAN% 0 Element
1.1% 250Ki NAN% 0 Export
0.5% 126Ki NAN% 0 Data
0.5% 119Ki NAN% 0 Function
0.1% 15.3Ki NAN% 0 Table
0.0% 50 NAN% 0 sourceMappingURL
0.0% 8 NAN% 0 [WASM Header]
0.0% 6 NAN% 0 Event
0.0% 4 NAN% 0 Start
0.0% 3 NAN% 0 DataCount
100.0% 22.8Mi 100.0% 0 TOTAL
The last file is the one that gets split and produces the main module with 165k export.