solidity-foundry-boilerplate
solidity-foundry-boilerplate copied to clipboard
chore: run medusa in CI
merge after #87
this PR aims to include a short medusa run in CI, to avoid pushing false positives/errors that can be caught within 5min of fuzzing
~~it's a WIP, and I'm currently torn between 3 approaches:~~ I believe having a docker image all our CI runs in is the way to go, see update below
docker-based action
this is a derivative of crytic/echidna-action, but modified to use eth-security-toolbox which has mostly up-to-date versions of a typical solidity toolchain, including medusa. with this, we don't have to worry about installing/updating medusa or crytic-compile, but have to (inside the action definition) deal with the mismatch between the github runner assuming everything will be executed as root and the image configuring everything for a normal user named ethsec
pros:
- all tools come in a neat package, dont have to install any dependencies
- can abstract away implementation details defined above so existing workflows dont need to be modified
- we'd be maintaining a dockerfile and scripts where we could add features of fix issues as the one described below
- configuration via action params is possible and would be very clean imo
cons:
- we could have issues due to tob not updating the tools as fast as we'd like (the image currently has a foundry version from july that doesnt recognize the newly added
forge fmtparam, so I had to add afoundryupin the Dockerfile ) - one more repo to maintain for the action
- having two sets of dev toolchains (the non-cointainerized we already use for most tests and the one inside the container) means we could have compilation/reproducibility issues in one but not the other, which will waste time and cause frustration
running the entire CI inside eth-security-toolbox
The entire CI workflow would run inside a container that already has most tools in it, so we could do away with most 'setup' steps in workflow definitions.
However, due to the root-vs-nonroot issue outlined above, we would have to either modify most ci steps so they walk around that or maintain a fork/extension of the aforementioned image so it makes everything available to root
pros
- low setup
- potentially better reproducibility (less left up to gh runner internal state)
cons
- have to deal with either ugly workarounds or maintaining a docker image of our very own
create an action to install medusa in the non-containerized gh runner environment
this would be similar to how we install foundry in CI: https://github.com/foundry-rs/foundry-toolchain/
pros:
- minimally disruptive
cons
- would probably take me a bit longer to implement
- doesnt help with other dependencies such as crytic-compile (which is required to run anything with medusa) -> workflows will end up being more verbose
update: I went for the 'run the entire CI inside a container' option, which proved simple enough if we maintain a fork of the docker image, see https://github.com/0xteddybear/eth-security-toolbox-ci/pull/1
TODO
- [x] use a non-personal container image
- [x] ~~set the timeout we think is reasonable (I believe for non-trivial projects should be 5min)~~ chose to set a runs # instead so running time is a red flag for high number of reverts
note: will update ci image version on monday, after the nightly tagged release