consensus-specs icon indicating copy to clipboard operation
consensus-specs copied to clipboard

Explore fuzzing pyspec functions

Open jtraglia opened this issue 9 months ago • 4 comments

It would be cool if we could fuzz individual spec functions somehow. This fuzzer could also be used to generate a corpus for clients to use in their own fuzzers too.

jtraglia avatar Mar 27 '25 13:03 jtraglia

Easiest way to start would be via black box fuzzing, then once the frequency of findings drops, switch to coverage based.

moodmosaic avatar Mar 27 '25 22:03 moodmosaic

This may also build on #4228:

--- a/scenario.yaml
+++ b/scenario.yaml
@@ -1,4 +1,6 @@
 scenario:
+  type: "function_fuzzer"
+  target_function: "process_attester_slashing"
   initial_state:
     validators:
       - {id: 1, balance: 32, active: true}
@@ -8,6 +10,11 @@ scenario:
       params: { proposer: "random" }
     - action: slash_validator
       params: { validator: "random" }
+  input_strategies:
+    attester_slashing:
+      strategy: "attester_slashing_strategy"
+      params:
+        validator_indices: [1, 2]
   invariants:
     - no_negative_balance
     - total_balance_properly_accounted

Inputs can be generated by Hypothesis strategies and have Hypofuzz run extended campaigns to find interesting cases (and then generate test vectors from them).

moodmosaic avatar Mar 30 '25 13:03 moodmosaic

CrossHair can be perhaps used for that.

ericsson49 avatar Apr 02 '25 16:04 ericsson49

@ericsson49, perhaps after catching the low hanging fruit first, because (I believe) CrossHair can hit path explosion when there is a lot of branching. (I assume it uses heuristics to tame the explosion but can you fully avoid it?)

moodmosaic avatar Apr 02 '25 16:04 moodmosaic