liboqs
liboqs copied to clipboard
open-quantum-safe
Code coverage
In all honesty, I don't have a tonne of experience with setting up code coverage in non-Golang environments.
I think that https://coveralls.io looks very nice and has a good interface. That said, I'm always a little bit wary to adopt SaaS when there could be a more "long lived" solution. (for example, choosing astyle over clang-format despite astyle existing on Sourceforge)
I was recommended gcov as well. I'm probably going to read the manual for gcov and then give it a go. If anyone has firm opinions and/or recommendations, I would be very interested to hear them.
Seems like a lot of C test coverage tools are proprietary.
Well, this is a bit later...
(included image is of lcov being run to generate code coverage)
Turns out it wasn't actually that hard. I was just being kind of daft.
I'm not 100% sure this actually makes 100% sense, so please do check me if something this diagnostic is showing seems inaccurate.
It seems like coveralls has a nice interface, but it's also just using gcov underneath.
So much for a diverse ecosystem of C code coverage tools! :)
Thanks for revisiting this. Is this output for a single execution of test_kem for a BIKE algorithm? Is it able to aggregate coverage across multiple runs?
Yeah it appears to be able to do a decent job of aggregating across multiple tests being run. I just need to figure out how to run all the tests haha.
Alright, looks like this is the approximate flow:
# build with --coverage flag
cmake -DCMAKE_BUILD_TYPE=Debug -DCMAKE_C_FLAGS="--coverage" -DCMAKE_CXX_FLAGS="--coverage" ..
# run files
<trying to sort this bit out>
# use lcov to automate running gcov for us
lcov --capture --directory . --output-file coverage.info
# generate an HTML report
genhtml coverage.info --output-directory coverage_html
Hi @aidenfoxivey! I took a stab at this a couple of months ago but haven't had time to properly see it through. Maybe my branch would be helpful to you?
