wasmtime icon indicating copy to clipboard operation
wasmtime copied to clipboard
verified publisher icon bytecodealliance

Update `wasmi` differential fuzzing

Open Robbepop opened this issue 3 years ago • 3 comments

Wasmtime has implemented differential fuzzing against the wasmi interpreter. However, the currently supported wasmi version is very old and outdated.

Benefits

The wasmi interpreter has since evolved quite a lot and could be helpful to more extensively fuzzy test against Wasmtime's feature set.

  • Fuzz test against modern wasmi versions instead of ancient one.
  • Being able to fuzz test against some Wasm proposals whereas the old wasmi only supported Wasm MVP.
  • Higher fuzz testing throughput since the newer wasmi versions are a lot faster.
  • New wasmi versions offer an API similar to the API of Wasmtime, therefore fuzz testing code should become slightly cleaner.

Downsides

The biggest downside is that the newer wasmi versions are not as battle tested as the old ones and therefore we might risk some false positives on the wasmi side which could introduce noise.

Implementation

I would open a PR myself if this issue is accepted. Shouldn't be too hard to update.

Alternatives

Stay on the current old wasmi version since it is quite stable and battle tested.

Robbepop avatar Aug 30 '22 09:08 Robbepop

Updating would be great! If you'd like to send a PR I'd be happy to review. If you've got ideas about how to improve the differential fuzzing interface feel free to throw those in as well!

alexcrichton avatar Aug 30 '22 14:08 alexcrichton

Subscribe to Label Action

cc @fitzgen

This issue or pull request has been labeled: "fuzzing"

Thus the following users have been cc'd because of the following labels:

  • fitzgen: fuzzing

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

github-actions[bot] avatar Aug 30 '22 14:08 github-actions[bot]

Updating would be great! If you'd like to send a PR I'd be happy to review. If you've got ideas about how to improve the differential fuzzing interface feel free to throw those in as well!

Cool, I will come up with a PR in the next days. I have not yet taken an in-depth look at the Wasmtime fuzzing interface but will do when I am working on it and maybe come up with improvements. :)

Robbepop avatar Aug 30 '22 15:08 Robbepop