covr icon indicating copy to clipboard operation
covr copied to clipboard

covr with `options(covr.record_tests = TRUE)` causes error when `.libPaths()` are modified mid-test-suite

Open dgkf opened this issue 2 years ago • 0 comments

Posting this primarily as a TODO item for myself, as this is related to code that I contributed.

Overview

Discovered when running covr::package_coverage with options(covr.record_tests) against package renv, it seems that accessing .libPaths() within the renv test is causing an error. This hasn't been a problem in the past, but operating on the assumption that library paths are not mutated throughout testing is perhaps too assumptive and may require broader revision to the way that test srcrefs are identified.

Error in `normalizePath(.libPaths()[[1]])`:
(converted from warning) path[1]="/tmp/Rtmpku41Ov/renv-test-83bf4993f94c/renv/library/R-4.1/x86_64-pc-linux-gnu": 
  No such file or directory 

Backtrace: 
1. renv:::test_that(...) at test-infrastructure.R:31:0 
2. renv:::stopf("test %s has corrupted libpaths", shQuote(desc)) at tests/testthat/helper-aaa.R:51:4 
3. covr:::count("utils-format.R:13:3:13:40:3:40:28278:28278") 
4. covr:::count_test(key) 
5. covr:::update_current_test(key) 
6. base::startsWith(syscall_srcfile[has_srcfile], normalizePath(.libPaths()[[1]]))

Proposed fix

As a first pass at this, the immediate error would be resolved by removing assumption that .libPaths() always has length >= 1 and add error handling when using normalizePath (I think mustWork = FALSE will be sufficient).

In the longer term, it might be necessary to capture the testing lib that covr kicks off the testing with instead of using .libPaths() as it exists throughout the testing execution. This would be a bit more involved, and I'm not sure off hand what would be the best way to propagate this library path to each count trace call. Possibilities include:

  • Passing the path to each trace call (not ideal as it would affect trace injection in a way that is unnecessary when covr.record_tests = FALSE - ie most cases)
  • Using an internal stateful environment similar to the .counters environment to log the package test installation library for reference when discovering test trace srcrefs. This is less invasive, but relies on more state being managed by the covr namespace.

dgkf avatar May 31 '22 21:05 dgkf