Recursive test_r()

[radbasa]

Changes

A unit test handler than can accept a vector of paths (a combination of files and directories) to run tests on.

test_r() defaults to running testthat::test_file() on all *.R files found inside tests/testthat/ directory and nested directories. test_file() is intelligent enough to not do anything (returns NULL) if it does not find any tests.

Ways of using test_r():

• test_r()
• test_r(paths = "tests/testthat/test-main.R")
• test_r(paths = "tests/testthat/logic")
• test_r(paths = c("tests/testthat/test-main.R", "tests/testthat/logic")

Removed R6. Now just a collection of functions.

In addition to paths, test_r() accepts:

• inline_issues - defaults to FALSE, switches where failures and skipped test messages are located. TRUE mimics testthat::test_dir() behavior.
• raw_testthat_output - defaults to FALSE which causes test_r() to return a single data.frame. TRUE causes test_r() to return a list of testthat::test_file() results which are lists containing data of test results.

Also added:

• duration report with a default min_time = 0.1

Not (yet) added:

• testthat encouragement message

~~Added a unit test handler that can run tests on:~~

• ~~single test file~~
• ~~single test dir (non-recursive)~~
• ~~recursive test dirs from a base path~~

A lot of code taken from testthat internal functions to have the same look and feel. May need attribution.

~~I am open to suggestions on how to write unit tests for this.~~ Unit tests written. May need more detailed tests (snapshots).

Addresses Issue #39

How to test

• unit tests for test_r() are available in tests/testthat/test-test_r.R. It uses tests/testthat/test_recursive as its sample test files and directories

• ~~rhino::test_r() - recursive run on all valid test-*.R files in tests/testthat, if there are no nested test folders, just runs the basic testthat::test_dir("tests/testthat")~~

• ~~rhino::test_r(recursive = FALSE) - equivalent to testthat::test_dir("tests/testthat")~~

• ~~rhino::test_file("tests/testthat/main.R") - equivalent to testthat::test_file("tests/testthat/main.R")`~~

Covr

• ~~Appsilon/covr will need to be updated to handle nested test folders~~ Done. Appsilon/rhino@feature/covr-support version 1.3.0.9003

[codecov-commenter]

Codecov Report

Merging #433 (29a5d59) into main (6696e81) will increase coverage by 26.94%. The diff coverage is 92.95%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@             Coverage Diff             @@
##             main     #433       +/-   ##
===========================================
+ Coverage   11.90%   38.85%   +26.94%     
===========================================
  Files           8        9        +1     
  Lines         294      435      +141     
===========================================
+ Hits           35      169      +134     
- Misses        259      266        +7     

Impacted Files	Coverage Δ
R/test_helpers.R	92.36% <92.36%> (ø)
R/tools.R	11.82% <100.00%> (+11.82%)	:arrow_up:
R/app.R	32.65% <0.00%> (+4.08%)	:arrow_up:

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[radbasa]

Hi Kamil,

Thank you for the code review.

Output

The output looks like that now because all we are doing now is running test_dir() multiple times. And we get the normal stdout output of test_dir(). What we can do is to use the SilentReporter of testthat and produce our own test result table. We can also have all the failures in one group. We will need to copy-paste more display code from testthat.

API

1. paths as a vector of files and directories is doable.
2. recursive = TRUE: We might want to run tests on only one directory, and not the nested directories.
3. ...: This is passed to test_dir() which can accept many other arguments. It also passes ... to testthat::find_test_scripts() which has its own set of arguments and also passes ... to testthat:::filter_test_scripts() and also finally passes ... to grepl().
4. test-.+\\.R$ filter: testthat::test_dir() when given a directory that does not contain at least one test-*.R file throws an error. The test_dir() filter argument matches file names after stripping test- and .R. It will only accept test-*.R files, unlike testthat::test_file() which will work with any file name. This is why when we go through the directories recursively we actually look for test-*.R files and get the unique directories.

Error in `testthat::test_dir()`:
! No test files found

5. getFromNamespace(): in a previous commit, I just copied over the code from testthat. I can revert to that.
6. fork and patch r-lib/testthat: @jakubnowicki 's requirement was to avoid modifying testthat and do everything in rhino.
7. But, we can still make rhino::test_r() return a data.frame, or a list of data.frames (the raw return output of testthat::test_dir() and test_file().
8. We can strip out the functions from the R6 class.

[radbasa]

Massive update

[radbasa]

Retracting this PR in favor of forking testthat itself https://github.com/Appsilon/testthat/pull/2