testthat icon indicating copy to clipboard operation
testthat copied to clipboard

Not accurate test counting in normal and Rmarkdown runs

Open Ahill25 opened this issue 2 years ago • 4 comments

Hey all, I have been trying to get an accurate test count on testthat package testing but seem to be running into faults. When running a test it may give numbers like shown below on the command line but when put into a dataframe the results are missing 26 tests. Where are these test numbers going? How is the result on the command line different from the actual variable assigned results? Thanks.

image

Ahill25 avatar Nov 14 '23 16:11 Ahill25

I don't really understand what you're asking here. Counting tests isn't an advertised feature of testthat.

hadley avatar Nov 29 '23 14:11 hadley

I don't really understand what you're asking here. Counting tests isn't an advertised feature of testthat.

The top line is test_package() being ran and assigned to a variable in an instance of R. The line under that is putting the variable into a dataframe. The 'passed' results in the dataframe come out to be 1520 and not 1544 as it says in command line.

Ahill25 avatar Nov 29 '23 15:11 Ahill25

https://github.com/r-lib/testthat/issues/1366#issue-856151829 This is a close answer to what I am trying to find but what if a test errors? I would have to manually count expectations I presume?

Ahill25 avatar Nov 29 '23 16:11 Ahill25

I don't ever use that functionality, so I don't have any insight, sorry.

hadley avatar Nov 29 '23 18:11 hadley