Deterministic output

[nickbabcock]

Description

The original use case of deterministic output was to facilitate snapshot testing of the generated typescript types in downstream projects. Discussion in https://github.com/wasm-bindgen/wasm-bindgen/pull/4738 broaden the scope to include all outputs, not just types.

Note even with this PR the underlying Wasm can still differ between builds with different codegen-units settings because how the compiler optimizes based on codegen-units will impact the Wasm output.

However, the interface (exports, imports, JS bindings, TypeScript definitions) is now fully deterministic.

This determinism is accomplished a few ways:

• hash suffixes (eg: __wbindgen_closure__destroy__h7a3b9c2d1e4f5678) are normalized with a sequential counter (like how what was previously done in test code): __wbindgen_closure__destroy__h0000000000000000
• Wasm exports are sorted by name and function signature
• JS imports and exports are sorted

The normalization relies on walrus's Type having a consistent ordering. So if we have an opinionated sort order, we can define our own, otherwise we can keep walrus's.

Closes #4738 https://github.com/wasm-bindgen/wasm-bindgen/issues/4737

Potential for discussion:

• All this sorting seems to cause a roughly a 100% increase in run-time. More investigation is needed.

Checklist

• [ ] Verified changelog requirement

[nickbabcock]

Thanks for the review, I finished up running PR on several internal codebases, and some more tweaks are needed, so I'm converting this to a draft just to avoid any accidental merging. Sorry for the confusion. We're close. 😄

[guybedford]

@nickbabcock I had no intention of stepping on your work here, but ended up working on a refactoring to the export system that had a deterministic effect in https://github.com/wasm-bindgen/wasm-bindgen/pull/4813.

It doesn't solve all determinism, but may solve some of the remaining issues you were hitting here.

[nickbabcock]

Thanks for looking out, but don't worry you won't step on my toes. I did hack up a solution that is yielding determinism on an internal codebase, but it isn't in shape to have a PR submitted from it 😄 https://github.com/nickbabcock/wasm-bindgen/tree/sort2

Don't let me be a blocker, I can incorporate your work as needed.

[guybedford]

https://github.com/wasm-bindgen/wasm-bindgen/pull/4813 has now landed which implements some of the changes here but not all of them.

Hopefully this should still rebase relatively cleanly to that though since they're in different code paths for the most part, to complete the non-deterministic output aspects.

Happy to review further and thanks for the efforts @nickbabcock.

[nickbabcock]

Thanks for the patience and sorry for some absenteeism. I have some time blocked off and am looking to get this PR back into shape

[codspeed-hq[bot]]

CodSpeed Performance Report

Merging #4800 will not alter performance

_{Comparing nickbabcock:sort (efe58de) with main (e58b99a)}

Summary

✅ 4 untouched

[nickbabcock]

@guybedford are you thinking that walrus::Module::emit_wasm should be updated as part of this process such that imports and exports are sorted prior to being emitted (similar to how ModuleTypes are now) 🤔

[guybedford]

@nickbabcock it could certainly be worthwhile to implement parts of this into Walrus, for example providing a sort function for Walrus or sort option, but I don't think we should change the default behaviour for backwards compatibility in Walrus itself.