linux-test-project
C++ branch coverage false positives
When using g++, there are many false positives with branch coverage, caused by standard operators and libraries, caused by if (ptr != NULL) checks employed by the compiler. Is there a way to ignore such false positives? False positives with printf is not persistent if we use a gcc.
Sample code
#include <cstdio>
#include <cstring>
#include <iostream>
#include <string>
int main(int argc, const char *argv[])
{
bool b = false;
if (strcmp(argv[1], "1") == 0)
b = true;
char *a = nullptr;
if (b)
printf("Hai\n");
delete[] a;
std::string str("asdads");
str = "cd";
std::cout << str << std::endl;
return 0;
}
Steps
$ gcc --version
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
$ g++ 1.cpp -fprofile-arcs -ftest-coverage -g -o 1 -std=c++11
$ lcov -c --rc lcov_branch_coverage=1 --no-external -d . -o 1.info
$ genhtml -s --rc lcov_branch_coverage=1 --legend 1.info -o o
Output
With GCC 4.8.5 on RHEL7
Yes.
See my earlier replies WRT ‘branch filtering’ – which is implemented in my version of the code. See “differential coverage” and the related descriptions and pull request.
@henry2cox your branch at https://github.com/henry2cox/lcov/tree/8728189f3e04a3efe0c2778de6c4a2e0cdb0d17d is unstable. I see incorrect result for the same input.
Reading data file 1.info
Found 8 entries.
Found common filename prefix "/usr/include/c++/9"
Writing .css and .png files.
Generating output.
processing dir /tmp/a
Processing file /tmp/a/1.cpp
lines=12 hit=12
processing dir /usr/include/c++/9/bits
Processing file bits/stl_iterator_base_types.h
lines=2 hit=0
Processing file bits/basic_string.h
lines=6 hit=0
Processing file bits/char_traits.h
lines=2 hit=0
Processing file bits/stl_iterator_base_funcs.h
lines=5 hit=0
Processing file bits/basic_string.tcc
lines=13 hit=0
processing dir /usr/include/c++/9/ext
Processing file ext/new_allocator.h
lines=2 hit=0
Processing file ext/type_traits.h
lines=2 hit=0
Writing directory view page.
Overall coverage rate:
lines......: 27.3% (12 of 44 lines)
functions......: 9.1% (1 of 11 functions)
branches......: 23.8% (10 of 42 branches)
I see a different result than you do...works for me.
$ g++ -fprofile-arcs -ftest-coverage -g -o foo -std=c++11 foo.cpp
$ ./foo 1
$ lcov --rc lcov_branch_coverage=1 --no-external -d . -o foo.info -c
Capturing coverage data from .
geninfo cmd: '/home/hcox/pwa/dspMethLibs/releng/coverage/lcov/bin/geninfo . --output-filename foo.info --no-external --rc lcov_branch_coverage=1'Found gcov version: 5.2.0
Using intermediate gcov format
Scanning . for .gcda files ...
Found 1 data files in .
Processing foo.gcda
Finished .info-file creation
$ ../../bin/genhtml -s --rc lcov_branch_coverage=1 --branch-coverage --legend foo.info -o rpt
Reading data file foo.info
Found 2 entries.
Found common filename prefix "/home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests"
Writing .css and .png files.
Generating output.
processing dir /mtkoss/gcc/5.2.0-rhel6/x86_64/include/c++/5.2.0
Processing file /mtkoss/gcc/5.2.0-rhel6/x86_64/include/c++/5.2.0/iostream
lines=1 hit=1
processing dir /home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests/foo
Processing file foo/foo.cpp
lines=13 hit=13
Writing directory view page.
Overall coverage rate:
lines......: 100.0% (14 of 14 lines)
functions......: 100.0% (3 of 3 functions)
branches......: 50.0% (10 of 20 branches)
Running again with branch filtering...
$ genhtml -s --rc lcov_branch_coverage=1 --branch-coverage --filter branch --legend foo.info -o rpt2
Reading data file foo.info
reading /home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests/foo/foo.cpp (for bogus branch filtering)
Found 2 entries.
Found common filename prefix "/home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests"
Writing .css and .png files.
Generating output.
processing dir /mtkoss/gcc/5.2.0-rhel6/x86_64/include/c++/5.2.0
Processing file /mtkoss/gcc/5.2.0-rhel6/x86_64/include/c++/5.2.0/iostream
lines=1 hit=1
processing dir /home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests/foo
Processing file foo/foo.cpp
lines=13 hit=13
Writing directory view page.
Overall coverage rate:
lines......: 100.0% (14 of 14 lines)
functions......: 100.0% (3 of 3 functions)
branches......: 50.0% (2 of 4 branches)
Filter suppressions 'branch':
6 instances
16 coverpoints
Nice to see the results. So here is what I did on my Ubuntu 20.04.1 LTS. Can you tell me what I'm missing?
2020-11-13 12:06:26 ~/code/foss
$ git clone https://github.com/henry2cox/lcov
Cloning into 'lcov'...
remote: Enumerating objects: 142, done.
remote: Counting objects: 100% (142/142), done.
remote: Compressing objects: 100% (70/70), done.
remote: Total 2201 (delta 81), reused 110 (delta 63), pack-reused 2059
Receiving objects: 100% (2201/2201), 1003.74 KiB | 7.22 MiB/s, done.
Resolving deltas: 100% (1559/1559), done.
2020-11-13 12:06:31 ~/code/foss
$ cd lcov
2020-11-13 12:06:35 ~/code/foss/lcov (master)
$ git co -t origin/diffcov_initial
Branch 'diffcov_initial' set up to track remote branch 'diffcov_initial' from 'origin' by rebasing.
Switched to a new branch 'diffcov_initial'
2020-11-13 12:06:45 ~/code/foss/lcov (diffcov_initial)
$ sudo make install
bin/install.sh bin/lcov /usr/local/bin/lcov -m 755 [5/28161]
bin/install.sh bin/genhtml /usr/local/bin/genhtml -m 755
(cd /usr/local/bin ; rm -f gendiffcov ; ln -s genhtml gendiffcov)
bin/install.sh bin/geninfo /usr/local/bin/geninfo -m 755
bin/install.sh bin/genpng /usr/local/bin/genpng -m 755
(cd /usr/local/bin ; rm -f gendiffpng ; ln -s genpng gendiffpng)
bin/install.sh bin/gendesc /usr/local/bin/gendesc -m 755
bin/install.sh bin/p4udiff /usr/local/share/lcov/support-scripts/p4udiff -m 755
bin/install.sh bin/p4annotate /usr/local/share/lcov/support-scripts/p4annotate -m 755
bin/install.sh bin/gitblame /usr/local/share/lcov/support-scripts/gitblame -m 755
bin/install.sh lib/lcovutil.pm /usr/local/lib/lcovutil.pm -m 755
bin/install.sh man/lcov.1 /usr/local/share/man/man1/lcov.1 -m 644
bin/install.sh man/genhtml.1 /usr/local/share/man/man1/genhtml.1 -m 644
(cd /usr/local/share/man/man1 ; rm -f gendiffcov.1 ; ln -s genhtml.1 gendiffcov.1)
bin/install.sh man/geninfo.1 /usr/local/share/man/man1/geninfo.1 -m 644
bin/install.sh man/genpng.1 /usr/local/share/man/man1/genpng.1 -m 644
(cd /usr/local/share/man/man1 ; rm -f gendiffpng.1 ; ln -s genpng.1 gendiffpng.1)
bin/install.sh man/gendesc.1 /usr/local/share/man/man1/gendesc.1 -m 644
bin/install.sh man/lcovrc.5 /usr/local/share/man/man5/lcovrc.5 -m 644
bin/install.sh lcovrc /usr/local/etc/lcovrc -m 644
bin/updateversion.pl /usr/local/bin/lcov 1.14 36.g8728189 1.14-36-g8728189
Updating bin tool /usr/local/bin/lcov
Done.
bin/updateversion.pl /usr/local/bin/genhtml 1.14 36.g8728189 1.14-36-g8728189
Updating bin tool /usr/local/bin/genhtml
Done.
bin/updateversion.pl /usr/local/bin/geninfo 1.14 36.g8728189 1.14-36-g8728189
Updating bin tool /usr/local/bin/geninfo
Done.
bin/updateversion.pl /usr/local/bin/genpng 1.14 36.g8728189 1.14-36-g8728189
Updating bin tool /usr/local/bin/genpng
Done.
bin/updateversion.pl /usr/local/bin/gendesc 1.14 36.g8728189 1.14-36-g8728189
Updating bin tool /usr/local/bin/gendesc
Done.
bin/updateversion.pl /usr/local/share/man/man1/lcov.1 1.14 36.g8728189 1.14-36-g8728189
Updating man page /usr/local/share/man/man1/lcov.1
Done.
bin/updateversion.pl /usr/local/share/man/man1/genhtml.1 1.14 36.g8728189 1.14-36-g8728189
Updating man page /usr/local/share/man/man1/genhtml.1
Done.
bin/updateversion.pl /usr/local/share/man/man1/geninfo.1 1.14 36.g8728189 1.14-36-g8728189
Updating man page /usr/local/share/man/man1/geninfo.1
Done.
bin/updateversion.pl /usr/local/share/man/man1/genpng.1 1.14 36.g8728189 1.14-36-g8728189
Updating man page /usr/local/share/man/man1/genpng.1
Done.
bin/updateversion.pl /usr/local/share/man/man1/gendesc.1 1.14 36.g8728189 1.14-36-g8728189
Updating man page /usr/local/share/man/man1/gendesc.1
Done.
bin/updateversion.pl /usr/local/share/man/man5/lcovrc.5 1.14 36.g8728189 1.14-36-g8728189
Updating man page /usr/local/share/man/man5/lcovrc.5
Done.
2020-11-13 12:07:08 ~/code/foss/lcov (diffcov_initial)
$
2020-11-13 12:08:11 ~/code/foss/lcov (diffcov_initial)
$ cd /tmp/a
2020-11-13 12:08:14 /tmp/a
$ lcov --version
lcov: LCOV version 1.14-36-g8728189
2020-11-13 12:09:36 /tmp/a
$ g++ 1.cpp -fprofile-arcs -ftest-coverage -g -o 1 -std=c++11
2020-11-13 12:09:38 /tmp/a
$ lcov -c --rc lcov_branch_coverage=1 --no-external -d . -o 1.info
Capturing coverage data from .
Found gcov version: 9.3.0
Using intermediate gcov format
Scanning . for .gcda files ...
Found 1 data files in .
Processing 1.gcda
Finished .info-file creation
2020-11-13 12:12:59 /tmp/a
$ genhtml -s --rc lcov_branch_coverage=1 --legend 1.info -o o
Can't locate DateTime.pm in @INC (you may need to install the DateTime module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.3
0 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/local/bin/genhtml line 86.
BEGIN failed--compilation aborted at /usr/local/bin/genhtml line 86.
2020-11-13 12:13:02 /tmp/a
$ sudo apt install -yqq libdatetime-perl
2020-11-13 12:13:26 /tmp/a
$ genhtml -s --rc lcov_branch_coverage=1 --legend 1.info -o o
Can't locate DateTime.pm in @INC (you may need to install the DateTime module) (@INC contains: /etc/perl /usr/local/lib/x86_64-linux-gnu/perl/5.30.0 /usr/local/share/perl/5.30.0 /usr/lib/x86_64-linux-gnu/perl5/5.3
0 /usr/share/perl5 /usr/lib/x86_64-linux-gnu/perl/5.30 /usr/share/perl/5.30 /usr/local/lib/site_perl /usr/lib/x86_64-linux-gnu/perl-base) at /usr/local/bin/genhtml line 86.
BEGIN failed--compilation aborted at /usr/local/bin/genhtml line 86.
2020-11-13 12:13:53 /tmp/a
$ sudo apt install -yqq libdatetime-format-w3cdtf-perl
2020-11-13 12:13:58 /tmp/a
$ genhtml -s --rc lcov_branch_coverage=1 --legend 1.info -o o
Reading data file 1.info
Found 8 entries.
Found common filename prefix "/usr/include/c++/9"
Writing .css and .png files.
Generating output.
processing dir /tmp/a
Processing file /tmp/a/1.cpp
lines=12 hit=12
processing dir /usr/include/c++/9/bits
Processing file bits/stl_iterator_base_funcs.h
lines=5 hit=0
Processing file bits/char_traits.h
lines=2 hit=0
Processing file bits/basic_string.h
lines=6 hit=0
Processing file bits/basic_string.tcc
lines=13 hit=0
Processing file bits/stl_iterator_base_types.h
lines=2 hit=0
processing dir /usr/include/c++/9/ext
Processing file ext/type_traits.h
lines=2 hit=0
Processing file ext/new_allocator.h
lines=2 hit=0
Writing directory view page.
Overall coverage rate:
lines......: 27.3% (12 of 44 lines)
functions......: 9.1% (1 of 11 functions)
branches......: 23.8% (10 of 42 branches)
Thanks for your help. Passing --filter branch option to lcov removes the branch false positives altogether. Good work.
However even with --no-external, genhtml produces coverage for external libraries. Let me dig a bit more.
I've raised https://github.com/henry2cox/lcov/pull/1 to fix the issue with --no-external. Please review.
Hi - I think I will let Peter add any update with respect to schedule - but the fixes described above (now in PR #169) will be merged "soon". :-) As discussed in other reports, please note that the filtering hack is something of a hack - and can be easily defeated by a bloody-minded user. You may also need to add 'noexcept' to some functions. With respect to the 'no external' issue mentioned above: I can't reproduce the issue :-( (works for me). Henry
I think this issue (bogus branch coverpoints) is addressed in SHA 5f659f63801e (as discussed above). I also completely agree that this approach is a workaround verging on ugly hack :-)
If you think that the issue is resolved: please close this issue. If it isn't fixed: please clarify what else should be done. Note that all real solutions probably need to be done within the compiler infrastructure. I don't think that lcov will ever have such infrastructure (though tools feeding or filtering lcov might).
Henry
Hey @henry2cox looks like your PR got merged. Congrats. Great work on that.
I'll test from the master branch of this repo when I get time and close this issue after that.
I see a different result than you do...works for me. $ g++ -fprofile-arcs -ftest-coverage -g -o foo -std=c++11 foo.cpp $ ./foo 1 $ lcov --rc lcov_branch_coverage=1 --no-external -d . -o foo.info -c Capturing coverage data from . geninfo cmd: '/home/hcox/pwa/dspMethLibs/releng/coverage/lcov/bin/geninfo . --output-filename foo.info --no-external --rc lcov_branch_coverage=1'Found gcov version: 5.2.0 Using intermediate gcov format Scanning . for .gcda files ... Found 1 data files in . Processing foo.gcda Finished .info-file creation $ ../../bin/genhtml -s --rc lcov_branch_coverage=1 --branch-coverage --legend foo.info -o rpt Reading data file foo.info Found 2 entries. Found common filename prefix "/home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests" Writing .css and .png files. Generating output. processing dir /mtkoss/gcc/5.2.0-rhel6/x86_64/include/c++/5.2.0 Processing file /mtkoss/gcc/5.2.0-rhel6/x86_64/include/c++/5.2.0/iostream lines=1 hit=1 processing dir /home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests/foo Processing file foo/foo.cpp lines=13 hit=13 Writing directory view page. Overall coverage rate: lines......: 100.0% (14 of 14 lines) functions......: 100.0% (3 of 3 functions) branches......: 50.0% (10 of 20 branches)
Running again with branch filtering... $ genhtml -s --rc lcov_branch_coverage=1 --branch-coverage --filter branch --legend foo.info -o rpt2 Reading data file foo.info reading /home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests/foo/foo.cpp (for bogus branch filtering) Found 2 entries. Found common filename prefix "/home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests" Writing .css and .png files. Generating output. processing dir /mtkoss/gcc/5.2.0-rhel6/x86_64/include/c++/5.2.0 Processing file /mtkoss/gcc/5.2.0-rhel6/x86_64/include/c++/5.2.0/iostream lines=1 hit=1 processing dir /home/hcox/pwa/dspMethLibs/releng/coverage/lcov/tests/foo Processing file foo/foo.cpp lines=13 hit=13 Writing directory view page. Overall coverage rate: lines......: 100.0% (14 of 14 lines) functions......: 100.0% (3 of 3 functions) branches......: 50.0% (2 of 4 branches) Filter suppressions 'branch': 6 instances 16 coverpoints
ubuntu 2020.04 lcov 1.14 Follow your operation，but result is not same could you tell me ，what`s wrong with me,please
wrong version. looks like you are using lcov 1.14 (very, very old). --filter option is much newer and is not yet available in any numbered release. You need to download latest version and install from github.
Just curious, how does this get downstreamed to different distros, say the next version of Ubuntu? Do you notify them or do they track and pull the latest?
Looks like you did not update to TOT (top of tree) - or you are not using the TOT version that you updated.
which genhtml might be instructive. genhtml --version might also be useful.
Similarly, just running /full/path/to/where/you/installed/genhtml ... would likely do what you wanted.