xls
xls copied to clipboard
google
Fuzz procs!
Now that we're starting to land procs, we should figure out the story for fuzzin' 'em. Might be a bit hard, but it definitely needs doing.
Procs add a couple new relevant features relative to functions: state and channels. Exercising state and channels will require generating a random number (0 or more) inputs for each channel, then reading out the all the values sent over each output channel. At each level (DSLX interpreter, IR interpreter, etc), the number and values of the outputs would be checked against each other. Currently the fuzzer only runs a single input through the generated function at at time so some work is necessary to add this.
To better exercise the interpreter, JIT, and generated control logic at the block/verilog level, the arrival time of inputs could be randomly varying. Above the block level this means the inputs arrive at different ticks. At the block/verilog level this means inputs arrive at different cycles. Also, at the block/verilog level randomly varying the cycles at which values are read from the output channels would also be interesting.
Initially, fuzzing could test a single proc at a time but ultimately we want to fuzz multiple procs. Networks of procs are interesting because they may deadlock. Some smarts would be necessary to ensure that the generated networks never deadlock OR perhaps the testing could check identical deadlocking behavior though that may be difficult to guarantee across the different abstraction levels.
Other useful features to have:
- non-blocking receives
- random FIFO depths (for proc networks)
Relevant bug on adding the necessary support to the Verilog simulation infrastructure: https://github.com/google/xls/issues/694
A couple other necessary/desirable features for proc fuzzing
- Extend crasher format to support description of values on inputs channels
- Improve proc support in the ir minimizer. This could be done iteratively as we find large samples which can't be reduced.