support for test coverage ?

[nikhedonia]

What is the best way to incoporate gcov or lcov into my buck build that has a gtest suite ?

[timmych]

Hi! All C++ coverage options get enabled by passing command line options to the compiler and/or linker, so all you need to do is read the compiler documentation and instruct Buck to pass the right arguments. For gcov/lcov, you'd pass -coverage to the compiler and to the linker. We use coverage mapping. https://llvm.org/docs/CoverageMappingFormat.html explains the flags that need to be passed to clang.

Next, after running the tests you'd have to extract the coverage data. How you do that depends on what type of coverage instrumentation was used. For coverage mapping, the same page (https://llvm.org/docs/CoverageMappingFormat.html) explains in the "Quick Start" section how to do it. Extensive documentation is also available for gcov/lcov. But all of this is pretty much beyond the duties of the build system.

[nikhedonia]

Let me refine my question: First, let's walk through the process of creating a coverage report for a simple foo.cpp and test.cpp:

Steps

1a) g++ -fprofile-arcs -ftest-coverage -c foo.cpp -o foo.o 1b) g++ -fprofile-arcs -ftest-coverage -c test.cpp -o test.o 2a) gcov foo.cpp 2b) gcov test.cpp 3) g++ -fprofile-arcs -ftest-coverage test.o foo.o -o test 4) ./test 5) gcov foo.cpp
6) lcov --capture --directory . --output-file coverage.info 7) genhtml coverage.info --output-directory html

What happens on each step?

1a) generates not only foo.o but also foo.gcno. 2a) generates foo.gcda. 4) runs test and generates foo.gcov and test.gcov 6) collects all gcov related files in . and generates coverage.info 7) creates a folder html with the coverage report

Challenges with Buck:

• where can I find *.gcno generated by the compilation of foo.cpp and test.cpp?
• how can I get all *.gcno` within a genrule?
• how can I collect all *.cpp of all my deps ?
• how can I collect all *.gcov *.gcda and *.gcno files required for step 6?

One very hacky way is by depending on the buck-out file structure and during a genrule cd $BUCK_PROJECT_ROOT. But is there a better way?

But all of this is pretty much beyond the duties of the build system.

Generating test-coverage reports are crucial for test-driven development and the build system should aid generation of those reports or at least enable users to express their intent easily. Where else would I describe build steps of an asset if not in my build system?

[SoftwareApe]

In Bazel there's the exact same issue. Bazel does support coverage with GCC, but hardcodes gcno/gcda/gcov everywhere. It seems simple. We'd just need a way to specify extra files by extension that should be transported and cached through the sandboxes, for compilation, for linking, for test execution, and most coverage tools could be supported.