bytecodealliance
wasi-preview1-component-adapter: A 64KiB arena is too small for environment variables
The single 64KiB page available to BumpArena in wasi-preview1-component-adapter is not reliably sufficient for allocating environment variables during initialization. This problem manifested for me as a crash in jco when running it inside a NixOS shell with a lot of environment variables set.
Thanks for the bug report. I'm not sure how easy this will be to fix, the component-adapter is a very change resistant bit of software, but I believe our original reasons for designing the long-lived BumpArena to have a capped size are no longer valid, so I'm going to see if I can change the design so that that it can grow unbounded.
@pchickey I was poking a bit at this today and wrote up a reproduction test case (ignore the commented out bits in the rest of Wasmtime).
Otherwise I think it should be possible to call cabi_realloc from the adapter. The one downside of that is that a single allocation can't be larger than 64k due to this constraint, but we can probably lift that at the same time perhaps. Most modules export a cabi_realloc anyway so they should avoid that linked code there too since they'd actually use the language's normal cabi_realloc.
thanks! I made a reproducer as well today but i like yours better. i was going through trying to decode the history of why we didnt always call cabi_realloc directly and trying to figure out, when it became possible to do so, did we choose not to do it or did we just not think about it? anyway I think I'm going kayaking tomorrow but I will give it a try wednesday.
Originally we assumed we couldn't call cabi_realloc at all and even now the import of cabi_realloc is a bit of a lie, sometimes it's hooked up to memory.grow. Supporting cabi_realloc itself was added in due to other issues down the road at some point.
Otherwise though I think we just forgot to handle the case of >=64k env/arg blocks...
I think I've got an idea, albeit a bit nasty in the implementation, of how to solve this so I'm going to give it a stab