bytecodealliance
jco componentize out of memory
Hey all, thanks for the cool project! I've been following various talks in the space for the last year and am really excited to be trying out some stuff!
Background
I'm working in a Rust codebase that's replacing a bunch of JavaScript. To accelerate the process, I'm looking at componentizing some of the JS modules.
The first project I picked is a bit of a doozy since, while the business logic is pretty straightforward and doesn't use too much craziness, the core is wholly dependent on some files on disk which consist of rules that drive things. This turns out to be fairly large; several megabytes worth of data uncompressed.
Here's my WIT file; it's pretty basic. Just take some input and spit out some output (JSON, but I'll deal with better types later).
package local:parser;
world parser {
export parse: func(input: string) -> string;
}
Initial approach
I could theoretically replace most portions of the library that currently rely on node fs and path APIs with WASI equivalents and hard-coded values respectively, but it seemed like lot less trouble to just bake these assets into the package. So, a few swings and a lot of jq later, I've transformed all of the file code into require('./data.js') essentially. It works!!
I've also figured out what seems to be a working rollup config and worked that into my tooling. There are now no external imports, and jco componentize gets past the first steps that otherwise failed when I had a dependency on fs or path.
The issue
When I run jco componentize, I get the following error:
Exception while evaluating top-level script
uncaught exception: out of memory
Additionally, some promises were rejected, but the rejection never handled:
Promise rejected but never handled: "out of memory"
Stack:
Error: the `componentize.wizer` function trapped
Caused by:
0: error while executing at wasm backtrace:
0: 0x7928c1 - <unknown>!<wasm function 12716>
1: 0x793234 - <unknown>!<wasm function 12732>
2: 0x204f5 - <unknown>!<wasm function 98>
3: 0x1f4b0 - <unknown>!<wasm function 96>
4: 0xa5bc - <unknown>!<wasm function 78>
5: 0x22a072 - <unknown>!<wasm function 5033>
1: Exited with i32 exit status 1
(jco componentize) Error: Failed to initialize the compiled Wasm binary with Wizer:
Wizering failed to complete
at componentize (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/node_modules/@bytecodealliance/componentize-js/src/componentize.js:305:13)
at async componentize (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/cmd/componentize.js:11:25)
at async file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/jco.js:213:9
Process finished with exit code 1
The full command, in case it's useful, is jco componentize --wit wit --world-name parser -o dist/parser.wasm bundle/parser.bundled.js.
I also tried it with the --aot flag and, after like 10 seconds, I got a similar error message.
I have a hunch that this is something to do with the size of the code, but I figured I'd ask the experts before spinning my wheels too much more, as I may have just missed something dumb.
Thanks in advance!
Quick update: after a bit of hacking around, I was able to get it to build a component. ~~I haven't had a chance to test if it WORKS yet in a host but that's progress!~~
The way I got it to build was by deferring the initialization. The few megabytes of structured data are still there, but I lazily do the initialization that processes them (ex: turning a dictionary with string keys into regex objects). So it seems the initial limit is something to do with pre initializing the WASM environment.
Is there a way to raise the limits? It would be perfectly acceptable for our use case to have a larger pre initialized memory :)
(The resulting wasm is pretty large btw around 45MB but that's fine for my use case.)
UPDATE 2: I was able to get it in a Rust host app, but ended up hitting a "wasm unreachable instruction executed" error 🤔 Unfortunately, per #537, I don't have a great way to debug this. Looking at the output of wasm-tools print, there are 5548 unreachable instructions 🤯
UPDATE 3: I eventually got it working under some circumstances, but with the following caveats.
My original approach to the port was pretty dumb and each "file" was just mapped into a JS object that held strings. Running this logic as part of the main init process caused an OOM. Reworking SOME of the processing logic (basically splitting the files into lines and then splitting each line on some delimiter and flattening into a single array) made the difference between being able to build the module or not.
However, the "lazy init" approach caused some issues; it would hit the unreachable instruction error, whereas I was able to eliminate this by doing the heavy loading into a "global" at the start of the JS entry point. This is probably a better idea anyways :) I just had to do a lot more work to avoid hitting either an OOM at jco componentize time or a confusing runtime error.
The other annoying part is that this baloons the final component size to 94MB 😅 For reference, the bundled JavaScript is around 10MB (most of this being data files).
In addition to what feels like some sort of memory limit during componentization, the other unsolved mystery is why AOT doesn't work. In the final working configuration, I get the following error with the --aot flag:
(jco componentize) ComponentError: Failed to decode Wasm
decoding item in module
Caused by:
magic header not detected: bad magic number - expected=[
0x0,
0x61,
0x73,
0x6d,
] actual=[
0xd0,
0x4,
0x0,
0x0,
] (at offset 0x0)
at componentNew (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/obj/wasm-tools.js:3618:11)
at componentNew (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/api.js:37:10)
at async componentize (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/node_modules/@bytecodealliance/componentize-js/src/componentize.js:419:5)
at async componentize (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/cmd/componentize.js:11:25)
at async file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/jco.js:213:9
This sounds like it might be an issue with weval, not jco, but recording the experience in case it's helpful.
I should be able to post this all up in a repo with an MRE sometime this week.
I apologize that this has turned into a bit of an "I'm not sure what's going on" issue; please feel free to split into sub-issues as you can identify them...
Here are some more confusing points which may or may not be bugs in jco directly...
Deterministic failure after multiple executions
After some number of executions in a loop using criterion from the Rust side, I end up hitting an unreachable instruction (it originates from my function, but the backtrace is unusable). If I fully instantiate a new component every time, the issue goes away, but I'm left with an unacceptable component instantiation delay :/
The weird (but I guess it should be comforting) part is that it's completely deterministic for a given wasm component build. Changing seemingly insignificant portions of the code that should not affect functionality can affect the number of iterations required before hitting an unreachable instruction.
I suspected there was a Store configuration issue preventing allocating a lot of memory or something, but instantiating a new component each time solves the issue, even when reusing the same Store.
I've also tried setting custom StoreLimits in case it was crashing due to hitting some limit, (setting instance, tables, and memories to 1 million, as well as setting trap_on_grow_failure), but that didn't produce any variation.
As such, it seems like it's some sort of bug.
AOT compilation
I was eventually able to get it compiled with --aot after adding a dynamic import vars plugin for rollup. Not really sure why this was required, as it does marginally work in the normal configuration.
Unfortunately, the AOT compiled version doesn't work. It hits an unreachable instruction. Leaving the plugin enabled doesn't change the behavior of the non-aot version.
Example to make things concrete
Repo that builds the component: https://github.com/stadiamaps/pelias-parser-component
`States` struct (basically from wasmtime examples)
struct States {
table: ResourceTable,
ctx: WasiCtx,
limits: StoreLimits,
}
impl States {
pub fn new() -> Self {
let table = ResourceTable::new();
let ctx = WasiCtxBuilder::new().build();
let limits = StoreLimitsBuilder::new()
.instances(1_000_000)
.tables(1_000_000)
.memories(1_000_000)
.trap_on_grow_failure(true)
.build();
Self {
table,
ctx,
limits
}
}
}
impl WasiView for States {
fn table(&mut self) -> &mut ResourceTable {
&mut self.table
}
fn ctx(&mut self) -> &mut WasiCtx {
&mut self.ctx
}
}
wasmtime component bindgen invocation
bindgen!({
path: "parser.wit",
world: "address-parser",
async: false,
additional_derives: [
serde::Deserialize,
serde::Serialize,
Clone,
Hash,
],
});
Wrapper to make invocation easier
pub struct Parser {
store: Store<States>,
// This is an attempt at speeding up failure recovery.
// Creating a new instance every time adds 7-10ms, which is less than ideal.
instance_pre: component::AddressParserPre<States>,
instance: AddressParser,
}
impl Parser {
pub fn init_with_engine(engine: &Engine) -> wasmtime::Result<Self> {
let start = Instant::now();
let component = Component::from_file(engine, "/path/to/pelias-parser-component/dist/parser.wasm")?;
eprintln!("Component init after {:?}", start.elapsed());
// Construct store for storing running states of the component
let wasi_view = States::new();
let mut store = Store::new(engine, wasi_view);
store.limiter(|state| &mut state.limits); // Custom limits config; probably not necessary
let linker = Linker::new(engine);
let pre = linker.instantiate_pre(&component)?;
let instance_pre = component::AddressParserPre::new(pre)?;
let instance = component::AddressParser::instantiate(&mut store, &component, &linker)?;
eprintln!("Instance init after {:?}", start.elapsed());
Ok(Self {
store,
instance_pre,
instance,
})
}
// The main entry point. Inline probably not necessary; doesn't seem to make a difference in benchmarks.
#[inline]
pub fn parse(&mut self, input: &str) -> anyhow::Result<ParsedComponents> {
// let instance = component::AddressParser::instantiate(&mut self.store, &self.component, &self.linker)?;
// let instance = self.instance_pre.instantiate(&mut self.store)?;
match self.instance.call_parse(&mut self.store, input) {
Ok(comps) => Ok(comps),
Err(e) => {
// Failure recovery path to recreate the component.
// Removing this will let you see which iteration the bench fails on.
let start = Instant::now();
self.instance = self.instance_pre.instantiate(&mut self.store)?;
eprintln!("Re-instantiated after {:?} due to {e:?}", start.elapsed());
self.instance.call_parse(&mut self.store, input)
}
}
}
}
Benchmark code
use std::time::Instant;
use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion};
use wasmtime::{Config, Engine, OptLevel};
use pelias_parser::Parser;
pub fn criterion_benchmark(c: &mut Criterion) {
let input = "30 w 26 st nyc 10010";
let mut config = Config::new();
config.cranelift_opt_level(OptLevel::Speed);
let engine = Engine::new(&config).expect("Unable to create engine");
let mut parser = Parser::init_with_engine(&engine).unwrap();
let start = Instant::now();
let mut iter = 0;
let mut failures = 0;
c.bench_with_input(BenchmarkId::new("parse", input), &input.to_string(), |b, i| b.iter(|| {
iter += 1;
let res = parser.parse(i);
if res.is_err() {
if failures == 0 {
eprintln!("#{iter} @ {:?} {res:?}", start.elapsed());
}
failures += 1;
}
}));
if failures > 0 {
eprintln!("{failures} / {iter} iterations failed");
}
}
criterion_group!(benches, criterion_benchmark);
criterion_main!(benches);
This out of memory error may be fixed in https://github.com/bytecodealliance/ComponentizeJS/pull/184. I'm running a new release cycle right now, do try it out on the latest versions when you can.
Thanks! That DOES sound related!
I updated ComponentizeJS and Wasmtime (to 30.0.1; running on macOS aarch64). It looks like the jco 1.10.0 release hasn't gone live on npm yet (CI failure?).
Also, in case anyone else is reading this issue later, I did eventually get minified JS that produces the correct output + simplified the build process using esbuild. The trick was to add --minify --keep-names; apparently without that, even running the bundle in node.js fails to produce the correct output 🤷♂ The linked repo is updated.
While JS size is no longer an issue (I've gotten it down to 5.9MB), repeated function calls on a component instance still eventually corrupt things to the point of executing an unreachable instruction.
Using the --aot flag for jco 1.9.1 + ComponentizeJS 0.17.0 now fails again at compile time (rather than runtime). But instead of a bad magic header like in the past, this time I get a stack overflow 😅
thread '<unknown>' has overflowed its stack
fatal runtime error: stack overflow
(jco componentize) Error: Failed to initialize the compiled Wasm binary with Weval:
Wevaling failed to complete
at componentize (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/node_modules/@bytecodealliance/componentize-js/src/componentize.js:268:13)
at async componentize (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/cmd/componentize.js:11:25)
at async file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/jco.js:213:9
Process finished with exit code 1
For the repeated calls error, that sounds like https://github.com/bytecodealliance/ComponentizeJS/issues/80.
Hey @ianthetechie just checking in here -- did jco v1.10.2 happen to fix your issue?
AOT compilation is back to crashing (magic header not detected), but I think the original OOM issue that I opened here is solved.
Feel free to close this issue or rename it to reflect the AOT compilation issues.
For AOT, the fix for this is to set the environment variable RUST_MIN_STACK to a larger value. We should include this in our Weval execution by default (@vados-cosmonic would be grateful if you could look into that!).
This works well by default - RUST_MIN_STACK: Math.max(8 * 1024 * 1024, Math.floor(freemem() * 0.1))
Hey @guybedford thanks for the note -- will make sure that's editable when we're doing AOT
OK, small PR up for this in Jco and Componentize-JS
https://github.com/bytecodealliance/jco/pull/608 https://github.com/bytecodealliance/ComponentizeJS/pull/206
Currently componentize reads the process ENV, so it SHOULD be possible to try this and confirm that it fixes the issue @ianthetechie , would you mind giving it a shot with RUST_MIN_STACK set like above?
Hmm.... I even updated my devDependencies to the latest:
"devDependencies": {
"@bytecodealliance/componentize-js": "^0.18.0",
"@bytecodealliance/jco": "^1.10.2",
"esbuild": "^0.25.0"
}
But when I am still getting an error with the --aot option. Here's my full build command: npm run build:esbuild && mkdir -p dist && jco componentize dist/parser.bundled.js --aot --disable all --wit wit -o dist/parser.wasm
$ RUST_MIN_STACK=8388608 npm run build
> [email protected] build
> npm run build:esbuild && mkdir -p dist && jco componentize dist/parser.bundled.js --aot --disable all --wit wit -o dist/parser.wasm
> [email protected] build:esbuild
> esbuild index.js --bundle --minify --keep-names --format=esm --outfile=dist/parser.bundled.js
dist/parser.bundled.js 5.6mb ⚠️
⚡ Done in 160ms
(jco componentize) ComponentError: Failed to decode Wasm
decoding item in module
Caused by:
magic header not detected: bad magic number - expected=[
0x0,
0x61,
0x73,
0x6d,
] actual=[
0x0,
0x0,
0x0,
0x0,
] (at offset 0x0)
at componentNew (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/obj/wasm-tools.js:3740:11)
at componentNew (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/api.js:37:10)
at async componentize (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/node_modules/@bytecodealliance/componentize-js/src/componentize.js:420:5)
at async componentize (file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/cmd/componentize.js:11:25)
at async file:///Users/ianthetechie/pelias-parser-component/node_modules/@bytecodealliance/jco/src/jco.js:224:9
Thanks for reporting this -- what's weird is that now you're getting a different error... would you mind not upgrading componentize-js to the latest?
I'm actually having some troubles updating componentize-js to the latest 0.18.0 in jco, which is why it's currently @ 0.17.0:
https://github.com/bytecodealliance/jco/blob/main/package.json#L27
There may be a new incompatibility with the componentize-js 0.18.0 right now, and getting a completely new error message might be related to that
A note here -- I was able to reproduce this, and it's certainly a componentize-js issue, in particular the interplay with --aot (the build passes without AOT).
@ianthetechie does disabling AOT for now work for you until we can find a lower level fix?
Confirming this is still broken on latest tooling, going to file this as an issue in upstream weval after some investigation
Weval is actually going to be removed from StarlingMonkey (e.g. https://github.com/bytecodealliance/StarlingMonkey/commit/f53dece2aacf09296301aa08ab28e4ac757d4817), so the official fix for this is going to be not using Weval, unfortunately.
Hopefully in the future we can re-enable it and get to the bottom of this bug in weval.