scoringutils
scoringutils copied to clipboard
Add additional documentation for `crps_sample()`, `logs_sample()` and `dss_sample()`
Update:
We wrap several functions from scoringRules
. This creates code complexity which I think is only warranted if we provide an extra benefit to the user. This benefit, in particular, could be more extensive function documentation. For example, we could add relevant formulas to the function documentation.
Old:
Currently we wrap several functions from scoringRules
, e.g. dss_sample()
, crps_sample()
, logs_sample()
.
The reason we wrote a wrapper for them is that function arguments in scoringutils have different names than the scoringRules
convention (e.g. we call arguments true_value
and predictions
instead of y
and dat
).
We could just call these functions directly, or reexport them.
How much of an issue is going to cause if we allow scoring rules that take different inputs?
I like the idea of developing a standard score class that takes a standard input (and that we can check etc) which would imply wrapping them is a good idea.
So currently, crps_sample()
is really just
function(observed, predicted, ...) {
assert_input_sample(observed, predicted)
scoringRules::crps_sample(
y = observed,
dat = predicted,
...
)
}
Our version has the added benefit of calling assert_input_sample()
, so we get standardised input checks. We also get arguments that are called observed
and predicted
. We could also add more extensive documentation on the CRPS (which is not included in the docs for scoringRules::crps_sample()
.
Just using scoringRules::crps_sample()
would slightly reduce code complexity.
I suggest we extend the existing documentation with all the formulas for the CRPS. This would be a clear benefit that could make it worthwhile having extra code. Otherwise, we should probably just get rid of the wrapper.
(Discussion about how we can check user-provided functions is here: #522)
I suggest we extend the existing documentation with all the formulas for the CRPS. This would be a clear benefit that could make it worthwhile having extra code. Otherwise, we should probably just get rid of the wrapper.
Agree