ModernCppStarter
ModernCppStarter copied to clipboard
Use `code-coverage.cmake` from `cmake-scripts`
Currently, the template just applies -fprofile-arcs -ftest-coverage
to the Greeter
target from test
. As far as I could tell, it doesn't really generate the code coverage data locally, and I think it'd be useful to be able to do that.
The code-coverage.cmake
script exposes targets that we can run if we want to generate the data locally too. It also does checks that the current ModernCppStarter solution doesn't.
~~I've been trying for an hour to get it to work and it seems target_code_coverage()
just doesn't find Greeter
...
Worth noting that it works with GreeterTests
.~~
~~Maybe you will have more luck?~~
Okay I figured out why the Greeter
target doesn't work. It's because target_code_coverage()
only generates the ccov-*
targets if the passed-in target is an executable one.
What is even weirder is that when generating the code coverage data, it includes doctest
in the report, but not Greeter
.
Okay, I don't know how I didn't see this before. It's so simple. To instrument a library target and an executable one, just call target_code_coverage()
on both. The library one won't get its custom ccov-*
targets, but the executable will. The report that cmake --build build/test --target ccov
now generates includes the greeter.cpp
file.
Although it also imports fmt
now along with doctest
. I'll try to figure out which directory to exclude so that it doesn't show up anymore.
Currently, the template just applies
-fprofile-arcs -ftest-coverage
to theGreeter
target from test. As far as I could tell, it doesn't really generate the code coverage data locally, and I think it'd be useful to be able to do that.
It should though, e.g. using Makefiles on Mac, the code coverage data is stored in <build_dir>/CMakeFiles/GreeterTests.dir/
. This is generated by simply calling the executable, e.g. after
cmake -Htest -Bbuild/test -DENABLE_TEST_COVERAGE=1
cmake --build build/test
./build/test/GreeterTests
To summarise the data you could simply call gcovr
.
> gcovr
------------------------------------------------------------------------------
GCC Code Coverage Report
Directory: .
------------------------------------------------------------------------------
File Lines Exec Cover Missing
------------------------------------------------------------------------------
source/greeter.cpp 8 8 100%
------------------------------------------------------------------------------
TOTAL 8 8 100%
------------------------------------------------------------------------------
TBH I don't do much local analysis myself and rely on platforms such as codecov.io for a nice visualisation. But all the data should be there and you should be able to easily generate the html reports locally.
I'm not really sure about the advantage of adding additional abstraction for the coverage, as the flags are straightforward to use and give most user control without additional required code.
To read more about gcov and how the flags work, see the wikipedia entry.
As far as I see it, the current solution doesn't support llvm-cov
? Also, the script makes it really simple and automate by exposing ccov
, ccov-all
and ccov-SpecificTarget
targets. It already comes with the cmake-scripts
dependency so it really isn't adding too much complexity. I made sure the previous option ENABLE_TEST_COVERAGE
remains there.
This doesn't seem to work anymore:
cmake -Htest -Bbuild/test -DENABLE_TEST_COVERAGE=1
-- CPM: Adding package [email protected] (v2.4.9)
-- CPM: Adding package [email protected] (v1.7.3)
-- CPM: Adding package CDT@ (/Users/adam/projects/cdt/test/..)
-- CPM: CDT: Adding package [email protected] (v1.8.0)
-- CPM: CDT: Adding package [email protected] (9.1.0)
-- Module support is disabled.
-- Version: 9.1.0
-- Build type:
-- CXX_STANDARD: 11
-- Required features: cxx_variadic_templates
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/adam/projects/cdt/build/test
cmake --build build/test
[ 37%] Built target fmt
[ 62%] Built target CDT
[100%] Built target CDTTests
./build/test/CDTTests
[doctest] doctest version is "2.4.9"
[doctest] run with "--help" for options
===============================================================================
[doctest] test cases: 2 | 2 passed | 0 failed | 0 skipped
[doctest] assertions: 5 | 5 passed | 0 failed |
[doctest] Status: SUCCESS!
gcovr
zsh: correct 'gcovr' to 'gcov' [nyae]? n
------------------------------------------------------------------------------
GCC Code Coverage Report
Directory: .
------------------------------------------------------------------------------
File Lines Exec Cover Missing
------------------------------------------------------------------------------
build/test/_deps/doctest-src/doctest/doctest.h
1995 679 34% 829,831,835,862,1024-1025,1029,1032,1036-1039,1045-1046,1078-1079,1123,1127,1152-1153,1256-1257,1300,1316,1318,1794,1801,1830,3238,3240,3283,3289-3291,3294-3296,3298-3302,3306-3307,3310-3311,3503,3506-3507,3510-3513,3516,3518-3521,3532,3563-3565,3593-3594,3602,3610-3615,3617,3619-3620,3622,3624,3626-3629,3631,3633-3634,3636-3638,3640-3642,3644,3646,3648,3650-3651,3653,3657,3669,3677-3678,3681-3684,3691,3695-3698,3701-3706,3709-3711,3714-3720,3723-3728,3741,3743-3744,3746-3748,3750,3754-3755,3758-3759,3762-3765,3779,3786-3789,3791,3793,3795,3797,3799,3801,3803-3808,3810-3811,3813,3819-3826,3832,3834-3840,3844,3849-3852,3855-3860,3863,3866-3868,3872-3875,3888,3890,3892,3894-3896,3898-3908,3915-3919,3926-3928,3936-3946,3948-3949,3955-3956,3963-3966,4034-4036,4038-4042,4044,4048-4050,4075-4077,4079-4087,4092,4108-4109,4113-4114,4116-4118,4121-4122,4125-4129,4131,4134-4137,4139,4143-4148,4150,4153-4157,4159,4161-4164,4166-4167,4169-4175,4177-4180,4189-4191,4193,4195-4198,4200,4204,4208-4209,4214,4217,4225,4273,4282,4284,4293-4299,4316,4319,4323-4327,4331-4335,4350,4352,4355-4359,4414-4416,4419,4421-4425,4429-4437,4466-4467,4473-4476,4478-4479,4482,4518-4521,4526-4527,4530-4532,4534-4535,4545,4547,4551-4553,4555,4746-4752,4755-4757,4823-4825,4829-4830,4832,4834-4836,4839,4841,4843,4864,4866,4873-4875,4880,4882,4884,4886,4888,4892,4899,4901,4905,4910,4913-4915,4917,4920,4922,4928-4930,4933-4937,4940-4942,4945,4947-4950,4953,4955,4958-4960,4963-4964,4967-4969,5012-5014,5040-5043,5062-5063,5078-5080,5082-5083,5085-5086,5088,5091-5093,5095-5096,5098-5099,5101,5104-5109,5114-5117,5119,5123-5127,5131-5132,5134-5135,5138-5139,5141-5142,5149-5151,5155-5157,5166-5169,5172,5174-5176,5181-5186,5191,5193-5195,5197,5199-5200,5204-5205,5207-5208,5213-5215,5218-5220,5236-5238,5241-5243,5246,5251-5253,5256-5263,5266-5269,5272-5277,5280,5282-5284,5287-5290,5293-5296,5299-5301,5304-5311,5313,5333-5336,5340-5341,5344-5347,5364,5366-5368,5370-5378,5383,5385-5390,5394,5397-5399,5402-5407,5409-5414,5421-5438,5440-5448,5450,5453-5454,5457,5463-5465,5468-5477,5480-5482,5484-5486,5488-5494,5496,5499-5501,5504,5506-5516,5518,5521-5522,5524-5526,5529-5534,5537,5539-5541,5543,5545-5549,5551,5553-5554,5556-5561,5563,5565,5568-5569,5571-5574,5576,5578,5580,5583-5587,5594-5595,5597-5598,5600-5611,5613-5614,5616,5618-5620,5623-5624,5626-5629,5632-5640,5642,5658,5660,5664-5665,5671,5675,5677-5678,5683-5684,5686-5687,5694-5695,5702-5703,5706-5709,5712-5716,5719-5721,5724-5726,5730-5731,5738,5740-5742,5744,5750-5751,5754-5755,5758,5760,5765-5772,5774-5775,5777-5782,5784,5786-5790,5793-5796,5799,5801-5802,5805-5807,5810-5813,5815-5816,5819-5822,5825-5827,5830-5831,5834,5836-5838,5840,5842-5844,5846-5848,5851,5853,5855-5858,5860-5864,5875-5876,5879-5882,5915,5917-5919,5922-5924,5927,5929-5930,5933-5936,5938-5942,5946,5950,5952-5954,5957-5959,5961-5969,5971-5973,5976-5980,5984,5986,6003-6005,6007-6016,6018-6020,6022-6036,6038-6112,6116-6117,6120-6127,6129-6131,6140,6142,6144,6162-6163,6165-6166,6168,6170-6175,6186,6226-6227,6232,6238,6241-6242,6245-6246,6249,6251,6253,6255-6256,6258-6259,6262,6267-6270,6272,6274-6275,6277-6278,6280-6286,6289,6292-6295,6298-6300,6307,6309,6311-6312,6314,6316,6319-6321,6323,6325,6327-6332,6335,6388-6389,6395-6399,6401,6435-6439,6441-6442,6444-6453,6455,6457-6462,6466-6467,6469-6470,6488,6491-6494,6496,6498-6499,6502-6505,6507-6509,6513,6526,6530-6533,6632-6633,6637-6638,6642-6643,6652-6653,6657-6658,6663,6666-6668,6672-6673,6677-6678,6682-6685,6689,6691,6693,6695,6707,6709-6711,6736,6739-6741,6753,6765,6783,6792,6794,6807-6811,6814-6816,6818,6820-6821,6823,6843,6846,6848,6850,6852,6854,6856,6864,6867-6869,6874,6884-6887,6889,6935-6937,6939,6945-6947,6951,6964,6983-6985,6988-6990,6998
build/test/_deps/fmt-src/include/fmt/core.h
52 52 100%
include/cdt/greeter.h 0 0 --%
source/greeter.cpp 8 8 100%
test/source/greeter.cpp 10 10 100%
test/source/main.cpp 0 0 --%
------------------------------------------------------------------------------
TOTAL 2065 749 36%
------------------------------------------------------------------------------
@acgetchell try a gcovr
config file too like this: https://github.com/ClausKlein/ModernCmakeStarter/blob/develop/gcovr.cfg
@ClausKlein That fixes it, thanks!