build-suite
build-suite copied to clipboard
kripken
Format for emscripten outputs
@titzer suggested that I upload builds from the emscripten test suite here, now that emscripten can output sexprs, so we would have full tests including libc usage, complete apps like python, freetype, etc. The current output format that I have been working on to get passing the test suite looks like this:
- The main JS file contains, as usual, the runtime support code, stuff that will be imported into the wasm module, the code that calls into the wasm module, etc. It doesn't contain the compiled code itself though.
- An asm.js file on the side, with the compiled code.
- The main JS file also contains the binaryen wasm.js polyfill. This will load the asm.js file, parse it into a wasm sexpr, then set up an interpreter for that wasm, which it hooks up to the surrounding JS environment.
For more direct testing of sexprs in this repo, I was thinking to upload builds that replace the asm.js with the sexpr translation (just by running the binaryen asm2wasm tool on them). So we would have
- Main JS file
- Sexpr file with the compiled code.
- As in #3 above, connection code/polyfill/etc.
I'm unsure what we want in #3. It seems like we should load the sexpr, then somehow check "is there native wasm support here". If there is, then give that native support the sexpr (or would it need to be binary?) and get a wasm module, or if not, then set up the interpreter as before, also receiving (what looks like) a wasm module.
cc @lukewagner @kg
ooc, are these builds that run in a browser and, if so, what wasm interpreter are you using?
I think I'm missing too much context to chime in on this question. Maybe I will understand it after I reread it a few more times. I don't know what binaryen actually is or what kind of testing you're doing, and it's not clear to me what wasm.js is or how it interacts with the wasm interpreter.
@lukewagner They can run in any JS environment, either browser or shell. The interpreter is binaryen's (wasm.js) (i.e., the binaryen interpreter compiled to JS).
@kg Sorry for the confusion. I was just remembering you said some things a while back about polyfill integration, and was curious what you thought about point 3, namely, what's a good way for this specific type of polyfill to ask the host environment "is there native wasm support here?" (and if there is, use it, otherwise, run in the interpreter as a polyfill). I don't have any code for that yet since I'm not sure what to do. Right now in my testing, it always goes into the interpreter polyfill.
OK, I see.
I think the approach we've been considering is for there to be a trivial way to check the DOM for any wasm support. We also need to expose a way to query specific features via the DOM, so you can decide in advance whether to use a wasm->js polyfill. Non-browser environments would need an equivalent to those two primitives. Does that make sense?
I don't know what those primitives will look like exactly, and I don't think we have any implementation out there that has them. @titzer might have added something like that to v8-native.
Sounds good to me. Once we have the details it should be easy to set things up in userspace.
In v8-native, you can just check for the global "WASM" object, which is exposed by V8, and it should have appropriate methods.
On Sat, Nov 21, 2015 at 6:41 AM, Alon Zakai [email protected] wrote:
Sounds good to me. Once we have the details it should be easy to set things up in userspace.
— Reply to this email directly or view it on GitHub https://github.com/WebAssembly/build-suite/issues/1#issuecomment-158592094 .
Ok, cool. Reading through the v8-native code, looks like what I want is WASM.instantiateModule, here is some code for that integration. Untested, but should be in the right area.
I've also finished with the emscripten test suite, binaryen passes the relevant core part (most everything but stuff we don't want like split memory). So I'm all set to upload some builds here for testing. I don't seem to be authorized to push to this repo though, I noticed now?
Side note: I worry WASM is short enough to be at risk for collisions with minified names or other acronyms in use on the web. Perhaps WebAssembly would be safer? But, bugs caused by such a collision should be easy to debug, so maybe it doesn't matter.
Reminder that I'm all set to upload some builds to this repo, but I need to be granted push access.
Whoops, you should be good to go.
On Sat, Nov 28, 2015 at 6:32 AM, Alon Zakai [email protected] wrote:
Reminder that I'm all set to upload some builds to this repo, but I need to be granted push access.
— Reply to this email directly or view it on GitHub https://github.com/WebAssembly/build-suite/issues/1#issuecomment-160250610 .
Ok, I uploaded two builds, hello world and zlib. Let me know if you want more now, or if first we could make sure these work properly with native wasm support - they work in the wasm interpreter, but I didn't test v8-native support, I'd be happy to work together on that.
Hello world uses syscalls, compiled printf and malloc, and so forth, so it by itself is a fairly complete testcase actually. zlib uses even more libc parts, and verifies that zlib works properly, so it's nice for checking correctness. It can also be used as a benchmark.
There is also now a readme in the repo, with instructions for how to run the builds (basically just run the .js file, but also details on the other files).
Ok, I'll try them out tomorrow and see if they work in v8!
On Sun, Nov 29, 2015 at 11:02 PM, Alon Zakai [email protected] wrote:
Ok, I uploaded two builds, hello world and zlib. Let me know if you want more now, or if first we could make sure these work properly with native wasm support - they work in the wasm interpreter, but I didn't test v8-native support, I'd be happy to work together on that.
Hello world uses syscalls, compiled printf and malloc, and so forth, so it by itself is a fairly complete testcase actually. zlib uses even more libc parts, and verifies that zlib works properly, so it's nice for checking correctness. It can also be used as a benchmark.
There is also now a readme in the repo, with instructions for how to run the builds (basically just run the .js file, but also details on the other files).
— Reply to this email directly or view it on GitHub https://github.com/WebAssembly/build-suite/issues/1#issuecomment-160475222 .
So, I tried out the helloworld example.
Here's what I found: in emscripten/hello_world/src.cpp.o.js, there is this code:
// Load the wasm module var binary = Module'readBinary';
// Create an instance of the module using native support in the JS engine.
var instance = WASM.instantiateModule(binary, {
"global.Math": global.Math,
"env": env
});
The first line should be a readbuffer() of the binary-encoded file (that output by sexpr-wasm). In d8 at least, "readbuffer" returns an ArrayBuffer of the contents of the. In the second line, the call to WASM.instantiateModule(), the second argument should be the FFI object; in this case, "env" works. The FFI object should be a flat JS object that has properties that are functions for the imports.
With these changes I get into decoding but crash due to a V8 decoder bug, which I am working on now.
Will keep you posted.
On Mon, Nov 30, 2015 at 7:37 PM, Ben L. Titzer [email protected] wrote:
Ok, I'll try them out tomorrow and see if they work in v8!
On Sun, Nov 29, 2015 at 11:02 PM, Alon Zakai [email protected] wrote:
Ok, I uploaded two builds, hello world and zlib. Let me know if you want more now, or if first we could make sure these work properly with native wasm support - they work in the wasm interpreter, but I didn't test v8-native support, I'd be happy to work together on that.
Hello world uses syscalls, compiled printf and malloc, and so forth, so it by itself is a fairly complete testcase actually. zlib uses even more libc parts, and verifies that zlib works properly, so it's nice for checking correctness. It can also be used as a benchmark.
There is also now a readme in the repo, with instructions for how to run the builds (basically just run the .js file, but also details on the other files).
— Reply to this email directly or view it on GitHub https://github.com/WebAssembly/build-suite/issues/1#issuecomment-160475222 .
Nice, this is getting close :)
About readbuffer, the emscripten code has this:
Module['readBinary'] = function readBinary(f) {
if (typeof readbuffer === 'function') {
return new Uint8Array(readbuffer(f));
}
var data = read(f, 'binary');
assert(typeof data === 'object');
return data;
};
so it should already use readbuffer on d8 properly, I think?
About the FFI object, doesn't it need the extra level? wasm imports are of a name from a module, so a single flat object seems like it would all be from one (unnamed) module. But I can flatten it out if that would be useful. What module name should I use in the wast?
I guess sexpr_wasm flattens the module names out, so maybe it doesn't matter?
On Tue, Dec 1, 2015 at 7:38 PM, Alon Zakai [email protected] wrote:
Nice, this is getting close :)
About readbuffer, the emscripten code has this:
Module['readBinary'] = function readBinary(f) { if (typeof readbuffer === 'function') { return new Uint8Array(readbuffer(f)); } var data = read(f, 'binary'); assert(typeof data === 'object'); return data; };
so it should already use readbuffer on d8 properly, I think?
About the FFI object, doesn't it need the extra level? wasm imports are of a name from a module, so a single flat object seems like it would all be from one (unnamed) module. But I can flatten it out if that would be useful. What module name should I use in the wast?
— Reply to this email directly or view it on GitHub https://github.com/WebAssembly/build-suite/issues/1#issuecomment-161057570 .
The call_indirect failing might be because I disabled the global initialization to get past it.
On Tue, Dec 1, 2015 at 7:41 PM, Ben L. Titzer [email protected] wrote:
I guess sexpr_wasm flattens the module names out, so maybe it doesn't matter?
On Tue, Dec 1, 2015 at 7:38 PM, Alon Zakai [email protected] wrote:
Nice, this is getting close :)
About readbuffer, the emscripten code has this:
Module['readBinary'] = function readBinary(f) { if (typeof readbuffer === 'function') { return new Uint8Array(readbuffer(f)); } var data = read(f, 'binary'); assert(typeof data === 'object'); return data; };
so it should already use readbuffer on d8 properly, I think?
About the FFI object, doesn't it need the extra level? wasm imports are of a name from a module, so a single flat object seems like it would all be from one (unnamed) module. But I can flatten it out if that would be useful. What module name should I use in the wast?
— Reply to this email directly or view it on GitHub https://github.com/WebAssembly/build-suite/issues/1#issuecomment-161057570 .
Looks like flattening should be enough for hello world, all imports are from env. But in zlib, there is an import from another module,
(import $f64-to-int "asm2wasm" "f64-to-int" (param f64) (result i32))
and other programs might import from Math. We can flatten that out into env for now, I pushed an update to the hello world build to this repo.
Any updates on the v8 side here? I'm interested if these builds can run now.