wasm4 icon indicating copy to clipboard operation
wasm4 copied to clipboard

Graphics fuzz test

Open aduros opened this issue 3 years ago • 2 comments

We should have a cart that draws random garbage to the screen using the different drawing functions (rect, line, oval, blit, etc) and compares each pass using a known precomputed hash of the framebuffer. If the hash differs it can trace something like FAILED on test #5393.

The main value of this, besides catching regressions, would be to ensure renderer parity across the web and C runtimes, as well as assist in developing 3rd party runtimes.

aduros avatar Aug 30 '22 20:08 aduros

I built a proof-of-concept thing in Zig a few months back that would embed the native runtime renderer into the WASM and compare it against whatever it's being ran on.

Basically there'd be functions that draw to the screen. It would run it both on the native runtime implementation and through the WASM-4 API, then compare the two framebuffers to see if they match. No hashing needed! It would have the advantage of being able to run thousands of tests through simple loops, basically fuzzing the graphics API.

~~Not perfect, but might spark some ideas for a more proper solution:~~ ~~https://github.com/JerwuQu/wasm4-gfx-test-cart/blob/master/src/main.zig~~ ~~https://github.com/JerwuQu/wasm4-gfx-test-cart/blob/master/src/tests.zig~~

Late edit:

I made something that's starting to be pretty complete: https://github.com/JerwuQu/wasm4-test-cart/

JerwuQu avatar Sep 02 '22 17:09 JerwuQu

Oh nice, yeah, that sounds like an even better idea!

aduros avatar Sep 06 '22 00:09 aduros