feat: remove different testing modes

[0xPhaze]

I've seen a few users getting confused as to why echidna is telling them that there are no tests to be found. It's usually the case that they did not include --test-mode assertion.

Suggestion: Remove different testing modes. Assertion mode should be the default. The normal mode (where functions with signature "echidna*() returns (bool)" are being used can just be included, i.e. if Echidna finds functions that meet that requirement, use the normal mode on those. Optimization mode could be done automatically for functions that return a comparable value (uint, int), i.e. "echidna*() returns (uint)". This might remove some of the friction for onboarding new users.

[ggrieco-tob]

We can also make testing mode mandatory. IMHO it is better make to sure we know how the user wants to test function that make a guess that could fail for some reason (perhaps producing no invariant failures and giving a false sense of security)

[0xPhaze]

I'm not sure I exactly understand what you mean. Imo the danger of a user getting a false sense of security would come from accidentally selecting an incorrect testing mode, which is why I would suggest to remove the need of specifying a testing mode.

I think assert-statements are pretty clear in their intention (i.e. this condition should always be true). Are you then suggesting to remove assertion mode altogether? I definitely enjoy the convenience of being able to add assert statements directly in the code without requiring a complex test-setup and/or create getters.

[0xKarl98]

I agree that we should make assertion mode by default , instead of adding it into yaml file explicitly .

[ggrieco-tob]

Imo the danger of a user getting a false sense of security would come from accidentally selecting an incorrect testing mode, which is why I would suggest to remove the need of specifying a testing mode.

If a user is using assertions and then they suddenly add one echidna_ function (let's assume that the function never returns false), they can fail to see the that the mode changed. This is particularly dangerous when using CI where users will not check for each execution, otherwise, detecting it looking at the UI should be easy.

[0xPhaze]

I'm suggesting that there just is no different mode. Assertions should be validated as with in the assertion mode and if one echidna_ function is added, then that should additionally be checked in the way that it is normally checked in.

[ggrieco-tob]

Oh, that's different thing.

[arcz]

I'm always in favor of simplifying the usage of tools and I think this is an interesting idea. The issue with having assertion mode always enabled is that the way it is currently implemented makes fuzzing slower compared to using the "normal" mode.

[arcz]

I've been thinking about it recently and this is what we can do right away:

• Change exploration mode so it happens automatically if no tests are detected. Have a CLI switch like --explore to ignore any tests to just collect coverage (should be a bit faster). Alternatively, use filters.
• If no tests are detected, default to exploration and let the user know it just collects coverage. Inform about the assertion mode in case this was not expected.
• Remove the optimization mode and let run optimization tests along the boolean tests. There is no technical limitation to this in Echidna, so I don't see why it needs to be a separate mode. This can be regulated with filters anyway.

I believe the rest of the testing modes can be cleaned up similarly but I didn't look into that yet.

[0xPhaze]

Are you differentiating exploration and assertion mode? I would have assumed that the default should just be assertion mode. Perhaps if no assertions are detected then it just becomes exploration mode (is there even a need to differentiate here, aside maybe from the final output format). But yeah, it would be great if echidna could just infer the type of test from the test return value.