WebAssembly
Use wasm customSections, to fix Binaryen WASM to text format round tripping failing for side module
I have been investigating building a dynarec for an emulator using dynamic linking. The hope was that i could manually modify WASM text and compile the side module using binaryen.js on the fly in a service worker.
As a POC i modified a emscripten unit test to: [Listing1] main.c
#include <dlfcn.h>
#include <stdio.h>
#include <emscripten.h>
void next(const char *z) {
void *lib_handle = dlopen("library.wasm", 0);
typedef void (*voidfunc)();
voidfunc x = (voidfunc)dlsym(lib_handle, "library_func");
if (!x) puts("cannot find side function");
else x();
}
int main() {
puts("hello from main");
emscripten_async_wget("library.wasm", "library.wasm", next, NULL);
puts("async");
return 0;
}
and library.c
#include <stdio.h>
void library_func() {
#ifdef USE_PRINTF
printf("1 hello from library: %p\n", (int)&library_func);
#else
puts("2 hello from library");
#endif
}
built with: emcc main.c -s WASM=1 -s EMULATED_FUNCTION_POINTERS=1 -s RELOCATABLE=1 -s MAIN_MODULE=1 -O2 -o exe.html and emcc -s WASM=1 library.c -s SIDE_MODULE=1 -O2 -o library.wasm
and i get the expected output: hello from main async 2 hello from library
Using binaryen-1.37.38-x86_64-linux.tar.gz I then created a POC by converting library.wasm to text (via command line use of binaryen wasm-dis) and using a service worker:
[Listing2] service-worker.js
self.importScripts('binaryen.js');
var wast = `
//wast/wat text
`;
self.addEventListener('install', event => {
self.skipWaiting();
});
self.addEventListener('activate', event => {
self.skipWaiting();
});
self.addEventListener('fetch', event => {
let url = event.request.url;
if (event.request.url.startsWith(self.location.origin)) {
var index = url.indexOf('library');
if(index == -1) {
return fetch(event.request).then(response => {
return response;
});
} else {
try {
var module = Binaryen.parseText(wast);
var binary = module.emitBinary();
module.dispose();
var headers = {'Content-Type': 'application/wasm'};
event.respondWith(
new Response(binary, { headers })
);
}catch(e){
console.log(e);
}
}
}
});
However this did not work. error:
Assertion failed: need the dylink section to be first
It seems the text does not have all the information the binary file did. Taking the text output and building the WASM binary file from that and re-running [Listing1] showed the same error. The WASM binary file sizes are slightly different. Is this expected behaviour?
I should add I repeated the same with WABT: The WebAssembly Binary Toolkit, with the same result. So i guess this is not the right approach?
Interesting. I think what's going on here is that the dynamic linking conventions specify that that section needs to be first:
https://github.com/WebAssembly/tool-conventions/blob/master/DynamicLinking.md#the-dylink-section
and emscripten emits it first, but none of the tools preserve that. It seems like we could either fix the tools, or change the spec to not require it to be first. Changing the spec is probably better, since the reason we required it to be first was make it easy to parse - but wasm added a .customSections API that lets the JS loading code avoid needing to parse the binary.
So what I'd recommend here is updating emscripten's JS loading code to use that new API (in src/support.js; can search for that "must be first" error message). After doing so, we can just remove the requirement to be first from the spec document.
Round-tripping the dynamic linking section through text would require use of the custom annotations proposal. There is an annotations format for the dynamic linking section in particular that we would probably want to use: https://github.com/WebAssembly/tool-conventions/blob/main/DynamicLinking.md#text-format-of-dylink0.
