dfhack
dfhack copied to clipboard
C++ code coverage support
DFHack has support for measuring coverage of Lua code, but has nothing for the C++ code. As we prepare DFHack for compatibility testing with the Steam version of DF, we'll need tooling to verify whether code is working properly on the new target.
I don't have enough knowledge for how to do this on Windows, but on platforms where we build with gcc (Linux and OSX), we can compile with -O0 --coverage to allow for gcov to generate coverage reports.
The workflow will be something like:
$ cd <dfhack src tree>/build
$ ccmake .. (and set COVERAGE to ON)
$ ninja install
$ dfhack
DFHack # test -- die (this will run all tests of all three types: 'none', 'title', and 'fortress' and then exit DF)
$ gcov (with options and file names to generate the annotated output)
I am unsure if the way dfhack causes the process to exit will allow the coverage data to be flushed to disk. If not, we might be able to manually call __gcov_flush() before exiting the process.
Once this basic workflow is functional, we can integrate it into Github Actions. I found https://docs.codecov.com/docs, which is a paid product, but can be used for free by open source projects. I'm still looking into this.
Code coverage for C++ in MSVC is only available in Enterprise Edition, and the cheapest legitimate source I can find for VSEE is $1100 (there are keys available for less, but I'm dubious of their provenance).
MSDN: Use code coverage to determine how much code is being tested
Ugh, yeah, Windows-based tools seem very not free. Here are some other, non-free options: https://stackify.com/code-coverage-tools/
This probably means that we'll only get coverage measurement on *nix, but the tests themselves should run on Windows just fine. That will be good enough to give us confidence that things are working, I think.