Difficulties with compiled code and testthat tests + how to turn off coverage of compiled code?

[SebKrantz]

Hi Jim, I'm trying to get covr to work for my C/C++ based package (https://github.com/SebKrantz/collapse) and facing difficulties which I'm not sure how to solve. I ran package_coverage(quiet = FALSE, clean = FALSE) and looked at the log, but essentially the line that fails is test_check("collapse"), which can also be seen on Travis-CI (https://travis-ci.com/github/SebKrantz/collapse) where is says a segmentation fault was encountered. Reading through the covr website, this could be perhaps because it was not possible to properly restart the R session before running the tests. I also noticed that running devtools::test() was problematic after the altered compilation with the -O0 --coverage flags: Sometimes the package fails to load, so the testing cannot start, and sometimes they run but all tests involving compiled code fail. Possibly the compilation itself is therefore problematic. I noticed that there are still -O2 and other default flags. This is an example of how one cpp file is compiled through package_coverage:

"C:/RBuildTools/4.0/mingw64/bin/"g++  -std=gnu++11 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG  -I'C:/Users/Sebastian Krantz/Documents/R/win-library/4.0/Rcpp/include'        -O2 -Wall  -mfpmath=sse -msse2 -mstackrealign -O0 --coverage -c fNdistinct.cpp -o fNdistinct.o

Having encountered these difficulties which I am not sure how to solve, I wanted to turn off coverage of all compiled code until I found a solution (which might be difficult since almost all of my functions call compiled code). I also failed to do that. My approaches were:

cov <- package_coverage(quiet = FALSE, line_exclusions = as.list(paste0("src/", list.files("src")))) This did not work, files were still compiled. Then I included a .covignore file in the root directory with the lines

src
/src/
*.cpp
*.c

which also yielded no response. Finally I tried to set options(covr.exclude_pattern = ".cpp") (which is probably not the intended use but anyway) - without success.

It would be great if you could let me know how to either get this running for all of my code or how to exclude my compiled code from the coverage testing. - presumably I'm not the only one with such issues.

Thanks,

Sebastian

[jimhester]

You should try compiling your package with -O0 outside of using covr and see if you also get failures. Sometimes code will work when optimized but fail without optimization. When this happens it typically means there is a bug in your package code that is being masked by the optimization.

You can do this by adding CXXFLAGS+=-O0 to your personal makevars file. Open it with usethis::edit_r_makevars().

The exclusion happen after the coverage is calculated, so changing any of those options will not affect this.

You can control what make flags your package is compiled with by covr with the covr.flags global option. It takes a named character vector of make variables and their settings. If you use options(covr.flags = c(CXXFLAGS = -O2, LDFLAGS="")) that should effectively turn off coverage for compiled code.

[SebKrantz]

Thanks for the response! What you say could be right, although I wonder if such a bug could have escaped CRAN's valgrind checks. Regarding compiling with CXXFLAGS+=-O0, I unfortunately am also not managing to have it overwrite the default flags, i.e. there is still an -O2 to the right of -O0 (this issue has been discussed here). Could you perhaps let me know how you get covr to insert those flags at the right end?

The other suggestion on how to turn off compiled code coverage worked perfectly, so thank you for that!

[jimhester]

The accepted answer in the linked issue tells you how to do it, you set CXXFLAGS+=-O0 in your personal makevars. usethis::edit_r_makevars() will open it for you if needed.

[SebKrantz]

Yes, but it adds it to the left, i.e. -O0 -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c, not -O2 -Wall -mfpmath=sse -msse2 -mstackrealign -c -O0, so -O2 takes precedence over -O0. But that's a tachnical issue, I thought perhaps you knew the answer.

[jimhester]

not in a properly configured system, something is wrong with your system configuration.