statefulness detector

[phi-go]

Add a statefulness detection step as part of coverage measurement to detect if the fuzz target is stateful. Compare the coverage measurement output by running the corpus in two different orders, if the target is not stateful there should be no difference.

This is a proof of concept and still needs to be integrated properly. Comparisons can be done by number of covered lines or number of executions for each region, though anything provided by the coverage measurement result should be feasible.

Here is a sample output:

/src/sudo/plugins/sudoers/toke.c: 6347:5   - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6347:9   - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6347:25  - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6347:26  -  8132 != 8131
/src/sudo/plugins/sudoers/toke.c: 6348:2   -  8132 != 8131
/src/sudo/plugins/sudoers/toke.c: 6348:18  - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6351:1   - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6352:5   - 657005 != 657004
/src/sudo/plugins/sudoers/toke.c: 6352:21  - 657005 != 657004
Target IS stateful.

[maflcko]

Just for reference, any project that has afl enabled should have a metric "stabilty" on https://oss-fuzz.com/fuzzer-stats. The statistic will tell how good or bad the stability of a given fuzz target is.

[phi-go]

Yes, fair point. However, to my understanding oss-fuzz mostly uses libFuzzer where this is not available. Nor is it easy to extract where exactly this instability happens for the afl variant, which seems quite useful if one wants to investigate the cause.