scoringutils

scoringutils copied to clipboard

The main source of this request is the following example where we use `add_relative_skill` to compute a relative absolute error, which generates a bunch of warnings from p-value computations which...

elray1

`forecast` and `scores` objects print output every time a column is changed

27

Consider this example: ``` a

nikosbosse

Add link to score definitions in `summarise_score()`

1

Would be nice to see what each score means when you print out the output of the function. This could be a simple print method that adds a simple "see...

jamesmbaazam

Just noting that this has been fixed in `{distributional}` so the optional dependency could be removed once that's on CRAN. _Originally posted by @sbfnk in https://github.com/epiforecasts/scoringutils/issues/917#issuecomment-2413363824_

nikosbosse

Make `get_metrics()` more customisable

1

We could add additional functions / additional arguments to `get_metrics()`. Consider something like this: ``` get_metrics( example_quantile, exclude = c("ae_median"), replace = c(wis, purrr::partial(wis, na.rm = TRUE), add_args = c(wis,...

nikosbosse

There is this nice diagram that plots forecast probabilities for targets that resolved yes vs. no. Ideally, a forecaster should assign lots of probability to targets that resolve yes and...

nikosbosse

Rethink `score()` and maybe make it not S3 anymore

2

From Sam's review on #940:

nikosbosse

We have a lot of functions like `check_input_binary()`, ... in addition to the `assert_input_binary()` etc. We used them a while ago, but we're wiser now and I think they can...

nikosbosse

Rename/refactor? `assert_dims_ok_point`

1

The function is used for both binary and point forecasts, so we should make that clearer in the name. In the spirit of having code for different forecasting classes more...

nikosbosse

Use nice illustration of the effect of increasing the mean on increasing average scores

1

We have this visualisation:  generated by this code: ``` library(data.table) library(dplyr) library(scoringutils) library(ggplot2) library(tidyr) library(patchwork) ## Real Data ex filter(model == "EuroCOVIDhub-ensemble") scores score() setnames(scores, old = c("dss", "crps",...

nikosbosse