Understanding multi-abi

[bingen]

I’m trying to understand and use multi-abi feature. Starting from the example, I modify it to look like:

contract A{
    bool public bug = true;
    function trigger_bug() public{
        assert(false);
    }
}
contract B{
    A public a;
    constructor() public{
        a = new A();
    }
    function echidna_test() public returns(bool){
        return a.bug();
    }
}

But then it passes, so it’s ignoring that trigger_bug function. Why? If I add this function to contract B:

    function echidna_test_2() public returns(bool){
        a.trigger_bug();
        return true;
    }

Then it catches the buggy assert. I don’t fully get the rationale here. Is this the expected behavior?

[gustavo-grieco]

Do you have checkAsserts enabled?

[montyly]

Hi @bingen.

Echidna will call all the contracts, but will currently look for properties, or failed assertions (if checkAsserts is enabled) only in the contract provided by the --contract ContractName flag.

[incertia]

To shed some light on why you are getting the behavior you see.

1. Echidna will, by default, not check for assertions in normal function calls.
2. Echidna will, by default, only expose the functions in tested contract for fuzzing.
3. Failed assertions in a test are considered test failures, because the result is an execution failure and not true.
4. The checkAsserts enables the reporting of assertion failures in normal function calls.
5. The multi-abi feature allows echidna to call the functions of any contract passed in on the command line, not just the one with properties.

[bingen]

I didn’t have it, but I added it and it still passes:

~/.local/bin/echidna-test examples/solidity/basic/multi-abi.sol  --contract B --config /tmp/multi.yaml
Analyzing contract: /home/bingen/workspace/crytic/echidna/examples/solidity/basic/multi-abi.sol:B
echidna_test: passed! 🎉
assertion in a: passed! 🎉

Seed: -3247530757343182007

Where multi.yaml is:

multi-abi: true
checkAsserts: true

According to 4 and 5, with both enabled, shouldn’t it catch the assert in trigger_bug?

Thanks!

[gustavo-grieco]

We have a PR #515 where we added a new feature to detect assertion failures in any contract using an event (AssertionFailed). Can you please test it to see if it covers this use case?

[bingen]

I tried, it doesn’t work. stack test gives:

    basic/multi-abi.sol:                   FAIL (12.02s)
      src/test/Common.hs:67:
      echidna_test passed

And again:

~/.local/bin/echidna-test examples/solidity/basic/multi-abi.sol  --contract B --config /tmp/multi.yaml
Analyzing contract: /home/bingen/workspace/crytic/echidna/examples/solidity/basic/multi-abi.sol:B
echidna_test: passed! 🎉
assertion in a: passed! 🎉

Seed: 3555132460785512990

[gustavo-grieco]

I think this looks like a bug, but we probably need to refactor mult-abi, after some discussion.

[gustavo-grieco]

This works in echidna 2.0:

contract A{
    event AssertionFailed();
    bool public bug = true;
    function trigger_bug() public{
         emit AssertionFailed();
    }
}
contract B{
    A public a;
    constructor() public{
        a = new A();
    }
    function echidna_test() public returns(bool){
        return a.bug();
    }
}

and the result:

$ echidna-test multi.sol --contract B --test-mode assertion --multi-abi
Analyzing contract: /home/g/Code/echidna/multi.sol:B
a():  fuzzing (44479/50000)
AssertionFailed(..): failed!💥  
  Call sequence:
    trigger_bug()

Event sequence: AssertionFailed() from: A@0xb4c79daB8f259C7Aee6E5b2Aa729821864227e84