test_invariant_shrink fails for ubuntu but works on mac

[CyMule]

Component

Forge

Have you ensured that all of these are up to date?

• [X] Foundry
• [X] Foundryup

What version of Foundry are you on?

forge 0.2.0 (ecddbdb 2023-02-17T00:11:42.4198088Z)

What command(s) is the bug in?

No response

Operating System

Linux

Describe the bug

Test Failure

cargo test test_invariant_shrink fails on Linux dev 5.15.0-60-generic #66~20.04.1-Ubuntu SMP Wed Jan 25 09:41:30 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux but works on Darwin dev.local 22.1.0 Darwin Kernel Version 22.1.0: Sun Oct 9 20:14:54 PDT 2022; root:xnu-8792.41.9~2/RELEASE_X86_64 x86_64

running 1 test
test invariant::test_invariant_shrink ... FAILED

failures:

---- invariant::test_invariant_shrink stdout ----
thread 'invariant::test_invariant_shrink' panicked at 'assertion failed: `(left == right)`
  left: `3`,
 right: `2`', forge/tests/it/invariant.rs:164:47

Workaround

If I change the fuzz seed from 102 to 100 then the test passes on my system.

#[test]
fn test_invariant_shrink() {
    let mut runner = runner();

    let mut opts = TEST_OPTS;
    opts.fuzz.seed = Some(U256::from(100u32));
    runner.test_options = opts;

The original commit for #2745 had the seed set to 100, it was changed to 102 in the next commit for #2724 and there is still a comment from the first commit explaining:

// `fuzz_seed` at 100 makes this sequence shrinkable from 4 to 2.

Solution?

Changing the seed to 100 did work on both ubuntu and mac for me, but I'm not sure if this is the case for everyone.

Maybe the test runner could be run multiple times over a range of seeds? This doesn't seem ideal...

Maybe a comment is added that this case might fail depending on your system? Or an additional message in the failure case?

    match counter {
        CounterExample::Single(_) => panic!("CounterExample should be a sequence."),
        // `fuzz_seed` at 100 makes this sequence shrinkable from 4 to 2.
        CounterExample::Sequence(sequence) => {
            assert_eq!(sequence.len(), 2, "This may fail depending on your system and fuzz seed.")

    };

Does anyone have a more optimal solution? I'm happy to help implement the changes.

Additional Context

As stated before, this test is originally added in #2745.

To my understanding, the test fuzzes InvariantInnerContract.t.sol.

The goal of the test is to find a "minimum representation" of the sequence to find an "invariant failure" for the contract. By starting with

opts.fuzz.seed = Some(U256::from(102u32))

on my system, the test runner is only able to find a sequence of 3 steps to break the invariant even though there is a shorter sequence of length 2, which is what the test expects.

[grandizzy]

@CyMule this should be fixed with https://github.com/foundry-rs/foundry/pull/7756 pls recheck @mattsse I think we can close it / reopen if hit again?

[grandizzy]

please reopen if you hit it again