foundry-rs
Foundry invariant tests not respecting max `runs` value
Component
Forge
Have you ensured that all of these are up to date?
- [X] Foundry
- [X] Foundryup
What version of Foundry are you on?
forge 0.2.0 (f9fc929 2023-06-29T00:04:55.475093000Z)
What command(s) is the bug in?
forge test --fuzz-seed 0
Operating System
Linux
Describe the bug
During a fuzzer comparative analysis, I encountered an unexpected issue with Foundry invariant tests.
In my tests, I am trying to measure the time foundry can catch an injected bug in Uniswap v2 with a set 24-hour timeout.
In order to achieve that, I have configured the runs value to its u32 max limit.
However, in some cases, the tests end prematurely, approximately 8 hours in, returning a "success" status. For example, using the seed 0 for this test case finished in 25960 seconds with status code 0.
Given the large value set for max runs ($2^{32}-1$, approximately 4 billion), it makes me believe that the max runs value is not being appropriately respected, or is being overridden at some point, but I have looked through the codebase and I could not find any explicit limitations.
Please let me know if I am doing something wrong, or how I can help investigate this further.
hey @aviggiano do you still have the output of that test, would like to see what value of Runs is displayed. If you still on for this comparison I'd be happy to debug issues you see. (there are also couple of perf improvements in latest builds that could make a difference). Thank you
Hello,
Sorry, not anymore. But I will try to reproduce this later and will share the output here.
Hey @aviggiano, thanks for raising this. Would be great to know if this is still an issue, especially given the improvements @grandizzy has made to fuzzing / invariant tests recently
Hey
I think we can close this issue for now, since I don't have the infrastructure to repro anymore. If I see this happening again, I can reopen it.