scoringutils icon indicating copy to clipboard operation
scoringutils copied to clipboard

Add additional documentation for `crps_sample()`, `logs_sample()` and `dss_sample()`

Open nikosbosse opened this issue 1 year ago • 4 comments

Update: We wrap several functions from scoringRules. This creates code complexity which I think is only warranted if we provide an extra benefit to the user. This benefit, in particular, could be more extensive function documentation. For example, we could add relevant formulas to the function documentation.


Old:

Currently we wrap several functions from scoringRules, e.g. dss_sample(), crps_sample(), logs_sample(). The reason we wrote a wrapper for them is that function arguments in scoringutils have different names than the scoringRules convention (e.g. we call arguments true_value and predictions instead of y and dat). We could just call these functions directly, or reexport them.

nikosbosse avatar Sep 15 '23 07:09 nikosbosse

How much of an issue is going to cause if we allow scoring rules that take different inputs?

seabbs avatar Oct 03 '23 09:10 seabbs

I like the idea of developing a standard score class that takes a standard input (and that we can check etc) which would imply wrapping them is a good idea.

seabbs avatar Oct 03 '23 09:10 seabbs

So currently, crps_sample() is really just

function(observed, predicted, ...) {
  assert_input_sample(observed, predicted)

  scoringRules::crps_sample(
    y = observed,
    dat = predicted,
    ...
  )
}

Our version has the added benefit of calling assert_input_sample(), so we get standardised input checks. We also get arguments that are called observed and predicted. We could also add more extensive documentation on the CRPS (which is not included in the docs for scoringRules::crps_sample().

Just using scoringRules::crps_sample() would slightly reduce code complexity.

I suggest we extend the existing documentation with all the formulas for the CRPS. This would be a clear benefit that could make it worthwhile having extra code. Otherwise, we should probably just get rid of the wrapper.

(Discussion about how we can check user-provided functions is here: #522)

nikosbosse avatar Jan 02 '24 12:01 nikosbosse

I suggest we extend the existing documentation with all the formulas for the CRPS. This would be a clear benefit that could make it worthwhile having extra code. Otherwise, we should probably just get rid of the wrapper.

Agree

seabbs avatar Jan 02 '24 23:01 seabbs