scoringutils

scoringutils copied to clipboard

- #608 - [ ] Create a new class - [x] #607 - [ ] create a class constructor - [ ] create a validator - [ ] Create methods...

nikosbosse

Define input format for categorical forecasts

5

As mentioned in #604, we would like to be able to score categorical/multiclass forecasts (or whatever the name should be, see #607. What should the expected format be? **Data.frame** Are...

nikosbosse

Add argument to control what kind of coverage is computed by `get_coverage()`

3

In #631 @sbfnk suggested adding an additional argument, `type = c("quantile", "interval") to `get_coverage()`. Below is a full proposal for what the function could look like. One open question is...

nikosbosse

Create print method for `scores()` objects

2

We currently have a `scores` object. This could get a nice print method analogously to the ones for forecast objects. It could for example print the scores that have already...

nikosbosse

Function could be simplified. Some code could be replaced by a method for a generic `sample_to_quantile.numeric()`. Function could use `interval_coverage_quantile()`.

nikosbosse

We would like users to be able to use metrics from other packages in `score()`. Sometimes, functions from those packages are different from our formats. - **0/1 input in binary...

nikosbosse

Documentation: Create a vignette for interactions with other packages

1

It would be nice to have a few showcases of how scoringutils interfaces with other packages - using functions from other packages - extending scoringutils - showcase functionality of other...

nikosbosse

Add additional documentation for `crps_sample()`, `logs_sample()` and `dss_sample()`

4

**Update**: We wrap several functions from `scoringRules`. This creates code complexity which I think is only warranted if we provide an extra benefit to the user. This benefit, in particular,...

nikosbosse

The vignette should explain - What input formats `score()` expects - What input format the different scores expect for the various forecast types - The workflow and how different functions...

nikosbosse

Create function `quantile_coverage_deviation()`

2

We have an `interval_coverage_dev_quantile()` function, but we don't have the equivalent for quantile coverage. New function would be `quantile_coverage_dev_quantile()`

nikosbosse