grain
grain copied to clipboard
feat(compiler)!: Custom Grain object files
Overview
This PR replaces Grain's intermediate .gr.wasm
files with Grain object files. This is for a variety of reasons, including:
- Debuggability. Our wasm-based linker is unable to reconstruct debugging/sourcemap information from wasm binaries. This approach once again generates proper sourcemaps. There is additional work to be done to make the sourcemaps truly usable, but this is a major step in the right direction.
- Optimizations. This change will allow us to do "whole program" optimizations to better assist Binaryen with its optimizations.
- Performance. Going from our IR to Binaryen IR and serializing is the bulk of compilation time. Since we no longer are serializing/deserializing many files, there is a performance boost. For a hello world program, cold compile time is down 10% and warm compile time is down 25%. When running in JS, cold compile time is down 15% and warm compile time is down 21%. This also gets subsecond warm builds in JS on my machine.
Technical details
Grain object files use the .gro
extension. As no wasm files are specific to Grain, the default output filename from the compiler uses the .wasm
extension rather than .gr.wasm
.
Object files follow this layout:
magic | version len | version | mash_program offset | cmi | mash_program |
---|---|---|---|---|---|
0xF0 0x9F 0x8C 0xBE | 5 | 0.6.3 | 1024 | <cmi> | <mash_program> |
mash_program
does include the module's cmi, but this format allows typechecking to unmarshal just the type information.
We use OCaml's Marshal
module as our binary format (and for Whole Grain, we can use our Marshal
module). js_of_ocaml can't marshal floats into a format that can safely be read back by native code. I wanted object files to be the same between native and js, so we store floats as int64
s in the mashtree to guarantee consistent behavior.
Reviewing this PR
The code delta for this PR is fairly small, but snapshot tests are now sexprs of mashtrees. As such, we have all new snapshots. I'd recommend clicking the first commit to review the code diff, or using the GitHub UI to hide changes to snapshots.
Closes #1903