ref-fvm
ref-fvm copied to clipboard
filecoin-project
Generate Code Coverage Reports
With Tests (https://github.com/filecoin-project/ref-fvm/issues/595, https://github.com/filecoin-project/ref-fvm/issues/585) being added to FVM, a good metric to start tracking is code coverage.
Note: this is separate from https://github.com/filecoin-project/ref-fvm/issues/314 that is for code coverage inside actors.
@dtynn pinging since this is (somewhat) related to the work you have already done
Hi @mriise I did some work in #314, but as you know, it focus on generating & collecting coverage instruments inside generated wasm bytecode..
For this issue and its related work, I do think filecoin-project/builtin-actors#22 & filecoin-project/builtin-actors#120 may be more helpful.
@dtynn yes, code coverage for most things can be generated, but for integraion tests we try to build WASM modules (mainly fil_ipld_actor is important here) that get then get ran by tests which would be nice to count towards coverage. We could just ignore the coverage that calls inside of the actors give and just run a custom setup for integration tests that prevents llvm-cov from trying to add instrumentation to the actors and just instruments the runtime.
Either way I would be happy to take up the work you have already done + add it to CI and get things merged.
ah, it would be nice to see some pieces of the work are useful to you., however, there are a few tricky steps before we can make everything work. let me know if you need any help
@mriise what's the current status of this? (i.e., what's done, what's left, etc.)
Current coverage (at the time of writing) is ~50% https://app.codecov.io/gh/filecoin-project/ref-fvm Done: https://github.com/filecoin-project/ref-fvm/pull/616, https://github.com/filecoin-project/ref-fvm/pull/612
easiest way to improve the % IMO would be to write future syscall tests in actors to excersize the sdk (currently at near 0%), syscalls, and everything else in the path. Most of the lines outside of syscall linking helping things that we should have covered are in DefaultKernel and the SDK.
(do builtin actors count for coverage??? dont think so...)
Ah, ok, so it sounds like we're including all the tests we care about in "code coverage".