echidna

echidna copied to clipboard

I am following this article to perform end-to-end fuzzing: https://github.com/crytic/building-secure-contracts/blob/master/program-analysis/echidna/end-to-end-testing.md I am currently using the binary version of Echidna 2.0.1 on Mac.

I did several simple examples and the fuzzing results are all correct. Currently I am trying to perform this method on pancake swap project with 4 main contracts. I have already seed the test through truffle and etheno.

Then when I run echidna, I am getting this "Error: running slither:" problem as the picture below:

I tried to change the solc version using the solc-select to 0.5.16 (which is the same solc version I adopt in the truffle project), but still failed. And I also tried to install and uninstall slither, but did not work either.

And another interesting thing I observed is that for the simple example I did, when I install slither analyzer and try to use echidna on the simple example, it actually failed with the same "Error: running slither". But when I uninstall slither, this problem can be solved for the simple examples, but the complicated example (pancake swap) still cannot work.

Any suggestions on this issue? Thank you!

I have attached my compressed truffle project for the pancakeswap fuzzing. The contract folder includes pancakeswap contracts (MasterChef, Router, Factory, Pancake Pair, two test token contracts) and the test contract E2E.sol. The script folder includes the script which seeds the contract states (run.js): pancake_fuzz.zip

And I also attach my initialize json file produced by Etheno and yaml config file: json and yaml.zip

Hi! Can you try running slither --print echidna --json - pancake/ and confirming if that fails? Can you also confirm which slither version you have installed? (slither --version should tell you)

Most often doing a npx hardhat clean fixes things for me when I get that error. It's kinda unsatisfying but it works :)

The truffle command should be something like truffle network --clean

Hi! Can you try running slither --print echidna --json - pancake/ and confirming if that fails? Can you also confirm which slither version you have installed? (slither --version should tell you)

Hi @elopez , thank you for commenting on this issue :) The first command I tried gives me the following output in my terminal, seems there's also error occuring:

And for the slither version, the output I get is 0.8.1:

For the install/uninstall slither I mentioned is the latest slither analyzer version: 0.8.3. So it seems I used to install slither before or this version comes from the Echidna binary file? May I know for the binary version for Echidna, whether slither is needed or it has already been wrapped in the binary file?

Most often doing a npx hardhat clean fixes things for me when I get that error. It's kinda unsatisfying but it works :)

The truffle command should be something like truffle network --clean

Hi @rappie , thank you for your suggestion. I tried this command but for the pancake project it still not works.

By the way, may I know how to use hardhat with echidna? Since hardhat can import multiple versions of solc compilers, so it is very convenient for projects with contract of different solidity pragma such as pancake swap. Truffle can only import one compiler version, so I am thinking to replace it with hardhat, but find few related documents online.

Hi! Can you try running slither --print echidna --json - pancake/ and confirming if that fails? Can you also confirm which slither version you have installed? (slither --version should tell you)

Hi @elopez , thank you for commenting on this issue :) The first command I tried gives me the following output in my terminal, seems there's also error occuring:

And for the slither version, the output I get is 0.8.1:

For the install/uninstall slither I mentioned is the latest slither analyzer version: 0.8.3. So it seems I used to install slither before or this version comes from the Echidna binary file? May I know for the binary version for Echidna, whether slither is needed or it has already been wrapped in the binary file?

In addition, currently I am using python 3.7, but the error I get when running the first command comes from the directory in python 3.9. I looked up this directory and I found two slither folders:

Then I looked up the same directory in python3.7, could not find any folder with the "slither" name. Is it the possible cause to this issue?

The correct way to run this is:

1. Enter the project directory: cd pancake
2. Run echidna: echidna-test . --contract E2E --config pancake.yaml

By the way, may I know how to use hardhat with echidna?

It just works out of the box. I think this is because echidna uses crytic-compile to compile the contract, which is able to detect the development environment. I believe it does npx hardhat compile --force in case of hardhat.

Hi! Can you try running slither --print echidna --json - pancake/ and confirming if that fails? Can you also confirm which slither version you have installed? (slither --version should tell you)

Hi @elopez , thank you for commenting on this issue :) The first command I tried gives me the following output in my terminal, seems there's also error occuring:

And for the slither version, the output I get is 0.8.1:

For the install/uninstall slither I mentioned is the latest slither analyzer version: 0.8.3. So it seems I used to install slither before or this version comes from the Echidna binary file? May I know for the binary version for Echidna, whether slither is needed or it has already been wrapped in the binary file?

Echidna does not bundle slither, you need to install it beforehand. You can try uninstalling slither on all python installations and then install the latest version. Alternatively, install the latest slither on a python venv and place it in front of the PATH

The correct way to run this is:

1. Enter the project directory: cd pancake
2. Run echidna: echidna-test . --contract E2E --config pancake.yaml

@ggrieco-tob Hi sir, I tried this comment, seems cannot work. I think for my case it should be ./echidna-test pancake/ --contract E2E --config pancake.yaml, since I am using the binary version of echidna, this command works on another similar example (but simpler)

Hi! Can you try running slither --print echidna --json - pancake/ and confirming if that fails? Can you also confirm which slither version you have installed? (slither --version should tell you)

Hi @elopez , thank you for commenting on this issue :) The first command I tried gives me the following output in my terminal, seems there's also error occuring: And for the slither version, the output I get is 0.8.1: For the install/uninstall slither I mentioned is the latest slither analyzer version: 0.8.3. So it seems I used to install slither before or this version comes from the Echidna binary file? May I know for the binary version for Echidna, whether slither is needed or it has already been wrapped in the binary file?

Echidna does not bundle slither, you need to install it beforehand. You can try uninstalling slither on all python installations and then install the latest version. Alternatively, install the latest slither on a python venv and place it in front of the PATH

Ok thank you for the suggestion, I will give it a shot. For now I have removed all the slither packages in my python3.7 and python3.9. I re-install the latest slither version (0.8.3) on python 3.7, but now when I try the simple example, it cannot work properly, giving the same error （slither running error)

It seems everything points to an issue in Slither but not Echidna. Can you open an issue in the slither github issue tracker?

It seems everything points to an issue in Slither but not Echidna. Can you open an issue in the slither github issue tracker?

Yes sure Sir @ggrieco-tob, I will open a issue there :)

And by the way, just asking, I see for contract dev tool such as hardhat and Ganache offer mainnet forking ability for ETH, BSC etc, so testers can test and interact on-chain contracts in a 'virtual' environment without going through the deployment process. So I am thinking maybe Echidna can add this kind of similar functionality? For researchers and testers, it would be very convenient for them to test some popular DeFi protocols on chain, all the contract states are already there, so simply writing some property tests are needed. What is your opinion sir? I would like to know if Echidna plans to develop similar functionality in the future :)

Im getting the same issue.

I think its because large contract which needs optmize args to compile dont get passed to slither.

I have a contract that without optmization uses too many evm slots because without optmize every variable is its own slot. With optimization enabled multiple variables can fit in 1 slot. Since i have alot of unit8 and unit32 variables etc this is needed to compile the contract.

running slither . errors running slither . --solc-args="optimize" succeeds

I was having the same error (echidna-test: Error running slither:) but it worked doing the echidna call like ~/echidna-test/echidna-test contracts/MyContract.sol --config echidna-config.yaml --contract MyContract.

With a echidna-config.yaml file with the remappings:

cryticArgs: 
  [
    "--solc-remaps",
    "@openzeppelin=node_modules/@openzeppelin/ 
    @rari-capital=node_modules/@rari-capital 
    @uniswap=node_modules/@uniswap 
    @chainlink=node_modules/@chainlink 
    hardhat=node_modules/hardhat"
  ]

Python3 version: 3.9.1 Python: 3.10.6 Slither: 0.8.3 Crytic-compile: 0.2.3 Echidna: 2.0.2

If it is of extra help, it worked when I installed py dependencies with pip3 install instead of pip install.

@BarrySauce

And by the way, just asking, I see for contract dev tool such as hardhat and Ganache offer mainnet forking ability for ETH, BSC etc, so testers can test and interact on-chain contracts in a 'virtual' environment without going through the deployment process.

The contract deployment is difficult to avoid in most of the cases (you will need to deploy something to check the tests at least), however, mainnet forking can be still very useful. We are considering this, but it requires some amount of refactoring. In the meantime, I'm closing this one.