Integrate TimHambourger/webassemblyjs-wasm-corruption test suite

[xtuc]

@TimHambourger made a very useful test harness, we could integrate it in our CI somehow (if @TimHambourger is ok with that).

Note we can keep a seperated repo.

[TimHambourger]

@xtuc Yes! I'm certainly ok with this.

I'm curious how you're imagining this. My first thought would be to make it part of the core repo such that it runs on make test and picks up the local versions of each package. Does Lerna have a notion of a non-publishing package or some such?

Also, thinking through things I like about the current harness:

• It's geared towards "real" input files from multiple sources. I could see gradually expanding that list, as well as the list of transforms.
• It walks through a full cycle of parse, edit, re-parse.
• It's pretty minimal: For each input + transform combo, it just cares whether the output is valid WASM or not. So it doesn't assert everything there is to know about the output, but it still can catch byte counting errors etc. that are likely to produce corrupt output.

I don't have a clear point here. Just figured stating those might help define how this harness can fit within the rest of the test suite.

[xtuc]

I would like to avoid adding more wasm binary into this repo and slow down current tests.

I can see two solutions:

• we could keep your repository (with a few structural changes), pull it from the CI and run it against the latest version of each packages.
• or a smoke testing approach, setup a cron in Travis and pull the latest version from npm of each packages.

FYI you can set private: true in the package.json to avoid Lerna to publish them.

[TimHambourger]

Yes, point well taken about not wanting to slow down current tests. Sure, I'm happy to keep this in my current repository, though I'd want to add you (and others as appropriate) as collaborators. I just don't want to become a bottleneck to changes getting approved.