foundry-rs
feat: add real-time counter for fuzz/invariant runs
dapptools has a counter like the one in the screenshot below so you can track progress in fuzz runs while those fuzz runs are executing. I recently ran a 10k fuzz run test that uses ffi, so it was quite slow and appears as if tests hang. A counter that updates live like the one below would be helpful for monitoring fuzz/invariant run progress and having confidence tests are still running as expected
I could see the run count, along with mean and median too - implemented in this line.
https://github.com/gakonst/foundry/blob/9de25d1928da362cd3b185ebaccc0e3d00aee2c8/forge/src/runner.rs#L101
Am I missing something here, or this feature is already implemented?
Oh, you are right.
Perhaps the request is probably moreso the real-time update of the counter (if supported by your terminal) than the count itself? Generally we should work on streaming in test results where possible which is (sort of) tracked in #141, so I'll close this as a duplicate.
Do let me know if something was misunderstood here!
Ah yea it does show the number of fuzz runs after a run already. The main motivation for opening this issue was to show a counter during the fuzz runs. I recently ran 10k fuzz runs with ffi, so it was quite slow, and it looks like your tests are hanging since there's nothing printed to the terminal during execution.
A real-time fuzz run counter seems to be a simpler scope than #141, and I don't see this functionality mentioned in there (though I only skimmed all the comments), so I feel like they could be tracked separately. But either way is fine with me 🙂
I agree it would be valuable to have some sort of real-time counter. Let's track it here then :sweat_smile: Can you update the original issue to reflect it?
This would also be useful for long-winded end-to-end fork tests that also fuzz inputs, since they can take a long time to run.
We should add coverage % for invariant runs. Basically, record locations of JUMPDESTs in a contract, and when we hit it record that and print out something like:
Runs: 6097
Coverage: 54% (27/54 Blocks)
