Add warning if tests do not use argument values other than the defaults

[katrinleinweber]

Hello!

As a Carpentries instructor, I'm considering to include a section about code/test coverage in a lesson for intermediate R users. I noticed that covr does not penalise tests which reuse a function's default argument values, rather than sending their own. I presume that this is intended, and that a dedicated argument/parameter value coverage may be a bit out-of-scope.

However, might covr users benefit from it checking whether any non-default values for function arguments with a default are used in the tests, and if so, showing a warning (or message)?

I imagine that this could help unit-testing-novices to understand that other coverages besides if & else branches can be important.

What do you think?

cc @cbeleites for suggesting covr::report() to me which got me thinking about this.

Edit: Hadn't finished drafting when I accidentally hit Submit...

[katrinleinweber]

In case the answer to https://github.com/r-lib/covr/issues/239 applies here as well, could you please summarise the reasoning? Are there alternative packages that can test parameter coverages?

[jimhester]

It would be pretty non-trivial to add support for this and I don't think it has a great deal of benefit, so the cost benefit is not high enough for me to work on this feature.

There is also as far as I know no way to include this information in the coverage reports generated by codecov or coveralls, so the only place this could be used is interactively.

I know of no alternative packages that test parameter coverages.

[cbeleites]

As we're talking R and thus statistics, I'm not sure how far this could be automated anyways: we'd need to be know the data type for that parameter plus possibly any narrowing in the value range (I'm thinking of parameters that specify, say, a percentile or a fraction or have other constraints that are arising from their meaning rather than the data type)

Already the data type is basically impossible to guess:

• a default of 0, is that 0.0 or 0L?
• a default of NULL doesn't tell anything about the expected data type
• think of indexing: a parameter that is used for indexing may be logical, integer or character: even if a default is given, it is not possible to conclude from that the required data type.

All in all, I'd say that parameter coverage is something that is very difficult to automatically check. But it is also something where semi-automated generation of assertions and unit tests may be a more practical way to achieve convenient coverage in practice.

[katrinleinweber]

Yes, parameter coverage could easily be too much work. But could the simpler facts

• "fewer , commas (and thus parameters), or
• "no , commas at all

appear in the testthat::expect_…ation inputs than in their function(…) definitions" be detected more easily, though?