Incorrect merge if custom tags are provided to `exclude-region`

[mp4096]

Dear Simon,

first, thank you for kcov!

I've run into a peculiar bug related to the issues discussed in #180 and fixed in cb43e13366d38e666c73fb7e3fd00af9f58b07ec.

Consider the following programs first.c and second.c, which are identical.

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
  if (argc == 99) {
    // LCOV_EXCL_START
    printf("Noooo...\n");
    // LCOV_EXCL_STOP
  }
  printf("Hello world!\n");
  exit(0);
}

Now I run

rm -rf first second combined *.out
gcc -g first.c -o ./first.out
gcc -g second.c -o ./second.out
kcov --version
kcov --exclude-pattern=/usr/lib first ./first.out
kcov --exclude-pattern=/usr/lib second ./second.out
kcov --merge combined first second

And get correct 100% coverage.

However, if I change LCOV_EXCL_START/LCOV_EXCL_STOP to custom tags like

#include <stdio.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
  if (argc == 99) {
    // start
    printf("Noooo...\n");
    // stop
  }
  printf("Hello world!\n");
  exit(0);
}

and run

rm -rf first second combined *.out
gcc -g first.c -o ./first.out
gcc -g second.c -o ./second.out
kcov --version
kcov --exclude-pattern=/usr/lib --exclude-region='// start:// stop' first ./first.out
kcov --exclude-pattern=/usr/lib --exclude-region='// start:// stop' second ./second.out
kcov --merge combined first second

then I get incorrect coverage (80 %) for the merged results, but correct coverage (100 %) for first and second separately (before merge).

Maybe it's because I use comments as tags?

I use Ubuntu 16.04 on x64 and kcov 35 from GitHub releases.

Thank you! :clap:

[SimonKagstrom]

Im not by the computer now, but I think this is because the custome exclude pattern isnt passed to the merge run.

The lcov exclude patterns are active by default, so they are always there. The options are not stored in the output metadata, so during --merge, it only uses the lcov patterns.

[mp4096]

Ah, I see.

Just to make sure -- I'm perfectly ok with using standard lcov patterns for my use case, just wanted to document / report this inconsistent behaviour.

I can close the issue if you think it's not worth the effort.

[SimonKagstrom]

We can keep it open. I think the real fix should perhaps be to store information about "ignored" lines in the output metadata, so that this can be picked up in the merge report instead of running the actual string matching there as well.

I guess you're the first to actually use region-exclusions plus merging together! Good analysis by the way! :-)

[mp4096]

You're right, I think there is no other way to generate correct merge reports... It just occured to me that somebody might want to use different tags for different files, something like

kcov --exclude-pattern=/usr/lib --exclude-region='// START:// STOP' first ./first.out
kcov --exclude-pattern=/usr/lib --exclude-region='// FOO:// BAR' second ./second.out
kcov --merge combined first second

So one should store either the --exclude-region and --exclude-line specs in the output metadata or the ignored lines.

[SimonKagstrom]

Yes, or simply not report the lines at all (which are ignored). I'll investigate and see what would be the best option here. At least it's probably not very likely to trigger for too many people :-)