rcmdcheck icon indicating copy to clipboard operation
rcmdcheck copied to clipboard

Print an account of test statuses

Open topepo opened this issue 6 years ago • 9 comments

When running R CMD check it would be good to know how many tests were skipped even if there are no failures.

topepo avatar Apr 23 '19 16:04 topepo

This is more of an issue for rcmdcheck I think.

gaborcsardi avatar Apr 23 '19 17:04 gaborcsardi

Specifically, I'd rejoice if this:

* checking tests ...
  Running ‘testthat.R’
 OK

got an upgrade to include more of this:

══ Results ══════════════════════
Duration: 14.7 s

OK:       390
Failed:   0
Skipped:  3

jennybc avatar Apr 23 '19 17:04 jennybc

OK, so this is actually a bit more involved, because parsing this information from the test results is not reliable, it depends on the reporter you are using.

Ideally, we would get the results directly from testthat, as an R object: https://github.com/r-lib/testthat/issues/875

Until that is done, I'll have to postpone this.

gaborcsardi avatar May 07 '19 08:05 gaborcsardi

Can we easily add extra data here? Maybe this should be done more from the testthat side? (i.e. what could test_check() emit to display extra info here?)

hadley avatar Sep 16 '21 11:09 hadley

I think that with the printing of the test results on GHA (and optionally locally as well), this is not so important any more. Locally you can always look at the .Rout file if you are interested. No?

gaborcsardi avatar Sep 16 '21 11:09 gaborcsardi

OTOH, we could still add some reporting to testthat, I think that would open up some new possibilties. E.g. testthat could write a summary of the results to a file in some format, if an env var is set?

gaborcsardi avatar Sep 16 '21 12:09 gaborcsardi

I think this is really just a duplicate of #121

hadley avatar Sep 16 '21 13:09 hadley

Yeah, I closed that one, this one is older and has more participants.

gaborcsardi avatar Sep 16 '21 13:09 gaborcsardi

I think this is still important; it'd be really nice to be able to see the number of skipped tests without having to dig down further.

hadley avatar Apr 26 '22 18:04 hadley