wasi-libc
wasi-libc copied to clipboard
WebAssembly
Investigate mimalloc
It'd be nice to provide alternative malloc implementations, with different code size vs performance tradeoffs.
See the comment here about mimalloc.
Hi!
I am interested in layout of Wasm linear memory on execution environment. As far as I know, with wasm-ld it should look as follows:
[ |0 ... data ... | ... stack ... | (__heap_base) ... heap ... (initial) ... (maximum)| ],
where __heap_base is a place where heap could be started and initial usually equals ceil(__heap_base) to 2**16, so initial >= __heap_base (or even >>).
Currently, wasi-libc uses dlmalloc with disabled mmap. And this dlmalloc starts with extending its top chunk with memory_grow. Execution environments execute this call as a memory "allocating" from initial to maximum. So it turns out that if allocator isn't explicitly support __heap_base space between __heap_base and initial isn't used. And this is up to 2**16 bytes - quite a lot. And current wasi-libc allocator doesn't support __heap_base. Moreover, wasm-ld in clang 9.0.0 doesn't export __heap_base and __data_end by default. Maybe I am wrong somewhere in this.
What do you think about it, does it make sense for WASI? Shall the old or a new allocator support __heap_base?
Yes, it does make sense to save this memory. I expect this could be done within the existing dlmalloc allocator, though I haven't looked at specifics. It would be useful for alternate allocators to do too.
wasm-ld still does support __heap_base and __data_end; it just doesn't emit them if they're not used.
@sunfishcode, I have made a fix, but It is inside dlmalloc's sys_alloc and I wanted to test it before making a PR somehow. Does wasi-libc have a test suite?
Great! For a testsuite, I have started working on setting up libc-test here -- for licensing issues we need to keep it in a separate repo. It's admittedly not a lot yet, but it's a start.
@sunfishcode As I understand you correctly, mimalloc shouldn't replace existing dlmalloc, but be an alternative (enable with some #define) allocator?
I imagine it would be a link-time alternative enabled via some kind of link flag.
Agreed; a link-time alternative is a good way to go here. So mimalloc would be built into its own .a file, named something like libc-mimalloc.a, installed alongside libc.a, so users can link with -lc-mimalloc or so.
Of course, if mimalloc works well enough, it may make sense to make it the default, but we can figure that out once we have something we can experiment with :-).
It sounds good, I will try to introduce __heap_base to mimalloc first and then add it to WASI.