artblocks-contracts
artblocks-contracts copied to clipboard
ArtBlocks
Add CI Coverage
Adds CI coverage via CircleCI + Coveralls integrations.
Closes #245
Per solidity-coverage docs here, Coveralls is the preferred coverage service (over something like codecov) due to its branch coverage reporting capabilities (which we use).
Interestingly, Coveralls actually is a bit more strict in branch coverage than our default solidity-coverage library reporting (which was used by us and our auditors for the V3 core contract). We may want some follow-on work to reach 100% coverage on the new Coveralls reports, mostly dealing with branch coverage on function modifiers.
Using a parallelized coverage pattern to speed up coverage. Bumped up solidity-coverage version to help prevent running out of memory during compiling contracts step on CircleCi's largest free-tier large instances (8gb memory), although updating the .solcover.js skipped directories to skip old minters and unused PBAB solidity files greatly improved that detail.
here is a current report - should probably do some tweaking to only report non-PBAB contracts, and the branch coverage looks a little odd... https://coveralls.io/builds/54646910
In the report what does this mean under "Run Details" 1740 of 4100 branches covered (42.44%)?
In the report what does this mean under "Run Details" 1740 of 4100 branches covered (42.44%)?
@lindsgil Currently, the percentage is quite low due to a bunch of redundant PBAB files being included in the coverage report. For branch coverage, if you have an if (x) {y} else {z} pattern, 100% branch coverage would be ensuring x evaluated as true and false in test suite. 50% coverage would be only evaluating x as either true or false, and 0% would be missing the statement altogether.
It does appear that coveralls picks up some branches on our solidity modifiers that our existing reporting doesn't pick up, so looking into that a bit. I expected to see 100% branch coverage on the V3 core contract (which is what us and our auditors measured), but coveralls highlights a few missed branches.