Unreachable revert path in invariant test

[nican0r]

Component

Forge

Have you ensured that all of these are up to date?

• [X] Foundry
• [X] Foundryup

What version of Foundry are you on?

forge 0.2.0 (551bcb5 2024-02-28T07:40:04.170939000Z)

What command(s) is the bug in?

forge test

Operating System

macOS (Intel)

Describe the bug

When running a invariant test with fail_on_revert = true I get a call sequence which causes an arithmetic error:

[FAIL. Reason: panic: arithmetic underflow or overflow (0x11)]
        [Sequence]
                sender=0x87b2D08110b7d50861141d7BBdd49326af3Ecb32 addr=[test/invariant/IonPool/ActorManager.t.sol:ActorManager]0x9334769e07df3B12e28D251c0DA84b72cfC0C1b6 calldata=borrow(uint128,uint128,uint128,uint128) args=[14234 [1.423e4], 10026 [1.002e4], 220588068670147766524049031157215289046 [2.205e38], 413]
                sender=0x00000000000000000000000000000000000000Bc addr=[test/invariant/IonPool/ActorManager.t.sol:ActorManager]0x9334769e07df3B12e28D251c0DA84b72cfC0C1b6 calldata=borrow(uint128,uint128,uint128,uint128) args=[340282366920938463463374607431768211452 [3.402e38], 2137374361896347860762549462890 [2.137e30], 340282366920938463463374607431768211452 [3.402e38], 1]
 invariant_meaningLessInvariant() (runs: 1, calls: 2, reverts: 1)

Looking at the second call trace with -vvv where the revert is supposed to be, there is no error:

I've added logging statements to the called borrow function that log different points in the function execution, including when the function call is complete and from the logs can see that both calls made by the fuzzer reach the end of their execution:

function borrow(uint128 borrowerIndex, uint128 ilkIndex, uint128 amount, uint128 warpTimeAmount) external {
        console.log("borrow is called");
        borrowerIndex = uint128(bound(borrowerIndex, 0, borrowers.length - 1));
        ilkIndex = uint128(bound(ilkIndex, 0, ionPool.ilkCount() - 1));

        borrowers[borrowerIndex].borrow(uint8(ilkIndex), amount, warpTimeAmount);
        console.log("borrow call completes successfully");
    }

It seems like the revert is due to an extra function call made by the fuzzer that isn't included in the call sequence which makes it impossible to reproduce with a unit test.

To rule out the possibility of a revert in the invariant I used the following invariant definition:

function invariant_meaningLessInvariant() external {
        console.log("this invariant does nothing");
    }

[grandizzy]

@nican0r there is an initial check that does not count as fuzz run where we assert the invariant in its initial state, and if it fails we exit early. So I am pretty sure the check went through OK (and printing the first sequence of logs), while first run resulted in revert, I think you could try reproducing by using sequence extracted from counterexample you posted above

ActorManager.borrow(14234, 10026, 220588068670147766524049031157215289046, 413)
ActorManager.borrow(340282366920938463463374607431768211452, 2137374361896347860762549462890, 340282366920938463463374607431768211452, 1)

[nican0r]

Hi @grandizzy, thanks for the response, I should've clarified that I actually had created a unit test with the call sequence that resulted in the broken invariant to try and figure out where the revert was (that's what's in the pic of the call trace) and was unable to reproduce it.

This was the main thing I was trying to highlight as I had tried this with many other call sequences that supposedly reverted and were identified by the fuzzer but none were reproducible so the corresponding unit tests provided no insight into where it might be reverting.

I've pushed the unit test to this repo and you can find it in the ActorManager.t.sol contract.

Running it with forge test --mt test_broken_call -vvvv shows no revert in the call trace. I haven't been able to isolate this issue into a minimum repro but I have had the same problem in another project so will link them here if it I can find again.

[grandizzy]

Awesome, thank you, please link the other example if you can find it. Re unit test there's also the possibility of vm.prank missing and influencing scenario (but not likely to be the issue here).

Is there a way I could reproduce the problem by running an invariant test on the repo you linked above? Please provide instruction if so, would really like to debug it

[grandizzy]

@nican0r I also see in the image you attached the vm.warp call which in invariant test was not preserved between calls until https://github.com/foundry-rs/foundry/pull/7219 This will be the default soon but until then there's a new setting preserve_state to be set to true in invariant section of config - maybe setting such makes a difference

[nican0r]

Awesome, the preserve_state config change worked! Really appreciate you highlighting this 🙏

[grandizzy]

No worries, it will be the default behaviour soon 👍