solidity-foundry-boilerplate icon indicating copy to clipboard operation
solidity-foundry-boilerplate copied to clipboard

chore: run medusa in CI

Open 0xteddybear opened this issue 1 year ago • 1 comments

merge after #87

this PR aims to include a short medusa run in CI, to avoid pushing false positives/errors that can be caught within 5min of fuzzing

~~it's a WIP, and I'm currently torn between 3 approaches:~~ I believe having a docker image all our CI runs in is the way to go, see update below

docker-based action

this is a derivative of crytic/echidna-action, but modified to use eth-security-toolbox which has mostly up-to-date versions of a typical solidity toolchain, including medusa. with this, we don't have to worry about installing/updating medusa or crytic-compile, but have to (inside the action definition) deal with the mismatch between the github runner assuming everything will be executed as root and the image configuring everything for a normal user named ethsec

pros:

  • all tools come in a neat package, dont have to install any dependencies
  • can abstract away implementation details defined above so existing workflows dont need to be modified
  • we'd be maintaining a dockerfile and scripts where we could add features of fix issues as the one described below
  • configuration via action params is possible and would be very clean imo

cons:

  • we could have issues due to tob not updating the tools as fast as we'd like (the image currently has a foundry version from july that doesnt recognize the newly added forge fmt param, so I had to add a foundryup in the Dockerfile )
  • one more repo to maintain for the action
  • having two sets of dev toolchains (the non-cointainerized we already use for most tests and the one inside the container) means we could have compilation/reproducibility issues in one but not the other, which will waste time and cause frustration

running the entire CI inside eth-security-toolbox

The entire CI workflow would run inside a container that already has most tools in it, so we could do away with most 'setup' steps in workflow definitions.

However, due to the root-vs-nonroot issue outlined above, we would have to either modify most ci steps so they walk around that or maintain a fork/extension of the aforementioned image so it makes everything available to root

pros

  • low setup
  • potentially better reproducibility (less left up to gh runner internal state)

cons

  • have to deal with either ugly workarounds or maintaining a docker image of our very own

create an action to install medusa in the non-containerized gh runner environment

this would be similar to how we install foundry in CI: https://github.com/foundry-rs/foundry-toolchain/

pros:

  • minimally disruptive

cons

  • would probably take me a bit longer to implement
  • doesnt help with other dependencies such as crytic-compile (which is required to run anything with medusa) -> workflows will end up being more verbose

0xteddybear avatar Oct 15 '24 17:10 0xteddybear

update: I went for the 'run the entire CI inside a container' option, which proved simple enough if we maintain a fork of the docker image, see https://github.com/0xteddybear/eth-security-toolbox-ci/pull/1

TODO

  • [x] use a non-personal container image
  • [x] ~~set the timeout we think is reasonable (I believe for non-trivial projects should be 5min)~~ chose to set a runs # instead so running time is a red flag for high number of reverts

0xteddybear avatar Oct 16 '24 16:10 0xteddybear

note: will update ci image version on monday, after the nightly tagged release

0xteddybear avatar Dec 20 '24 12:12 0xteddybear