bdk
bdk copied to clipboard
bitcoindevkit
Change configs for source-base code coverage
Description
This also changes the code coverage front end to coveralls instead of codecov, which had some issues with other changes in the PR. This will provide better and more accurate code coverage reports.
Notes to the reviewers
The tests run before generating the report are not exhaustive (not exhaustive earlier too, but I added as many as I could), and hence the report won't be 100% accurate.
Checklists
All Submissions:
- [x] I've signed all my commits
- [x] I followed the contribution guidelines
- [x] I ran
cargo fmtandcargo clippybefore committing
create coverage report in the cont_integration jobs itself to get coverage for the full test suit..
I actually tried that, but I couldn't get the reports from the separate (parallel) tests to merge, even though it should be possible according to the documentation for coveralls. I will try it some more and see if it works.
Maybe you could also upload the coverage report as an artifact so that it can be manually downloaded and inspected with a different tool if coveralls isn't accurate.
We have some code to upload artifacts in the "build nightly docs" job, you can take it from there.
In the last dev call we converged on approach of
- Update the existing coverage tool from codecov to better stuffs.
- Make the coverage test as exhaustive as possible..
Sorry, I thought it was gonna be easier to generate an html report from the coverage file, but it actually depends on many files in your current environment. I would suggest also generating a report and uploading that as well for convenience. You can do that with the genhtml command
In the last dev call we converged on approach of
* Update the existing coverage tool from codecov to better stuffs. * Make the coverage test as exhaustive as possible..
The problem with codecov was that it reported many extra lines (which are not present in the source file) which results in lower % coverage. I found coveralls to be more accurate.
The problem with codecov was that it reported many extra lines (which are not present in the source file) which results in lower % coverage. I found coveralls to be more accurate.
Yes I think this is what @rajarshimaitra meant, that's it's fine to switch to a new tool like grcov as you're doing.
With this PR looks like codecov report shows 90.87% coverage 🥇!
https://app.codecov.io/gh/wszdexdrf/bdk
Old config only showing 77.21%:
https://app.codecov.io/gh/bitcoindevkit/bdk
With this PR looks like codecov report shows 90.87% coverage 🥇!
https://app.codecov.io/gh/wszdexdrf/bdk
Old config only showing 77.21%:
https://app.codecov.io/gh/bitcoindevkit/bdk
I hate to say this, but that's not entirely correct. That was an older report I had generated while testing. That report has lesser number of total lines and hence it has higher percentage of coverage. The current version is https://coveralls.io/github/wszdexdrf/bdk Which is 79.18 % (81% if I can get it to remove examples from the report) :(
One good thing to note will be that even though the percentage is relatively unchanged, the number of tracked lines is almost doubled, so I think this should provide better tracking. And also, directly fixes the issue that some modules weren't being tracked...
Oh, I just noticed... You should change the badge in the README not to point to codecov anymore, but to point to coveralls
I just noticed on coveralls that it doesn't show the source code so there's no way to see what lines are covered or not, is there something you need to enable to get coveralls to show this info?
https://coveralls.io/builds/52019084/source?filename=src%2Fpsbt%2Fmod.rs
I just noticed on coveralls that it doesn't show the source code so there's no way to see what lines are covered or not, is there something you need to enable to get coveralls to show this info?
https://coveralls.io/builds/52019084/source?filename=src%2Fpsbt%2Fmod.rs
That's weird. It does show the code for me...
Edit: I found this. I guess we could use the HTML report for line by line coverage. I am waiting for any idea about how to proceed now.
Edit: I found this. I guess we could use the HTML report for line by line coverage. I am waiting for any idea about how to proceed now.
Ahh! OK I can confirm that once I login to coveralls with my github account I'm able to see the code and coverage details. As mentioned in the issue you found this is annoying, but I wouldn't say it's a show stopper.
@wszdexdrf looks like this needs another re-base to pickup new CI tests.