scoringutils
scoringutils copied to clipboard
epiforecasts
Utilities for Scoring and Assessing Predictions
I tried summarising by c("model", "quantile"), and specified metric = "interval_score", but still got all metrics back.
When some lower or upper bounds of a prediction interval (i.e. when some quantiles) are NA, then `score()` should alert the user somehow of this. Maybe the corresponding intervals should...
The function should give you an intuitive error message, when you try to pass the output form "summarise_scores" directly into "plot_correlations". `correlation` should give a warning when it filters out...
There is a black bar in the package [web documentation](epiforecasts.io/scoringutils). It looks like a dark theme issue. We see this in `epinowcast` so its likely linked to an interaction between...
It appears that there is a new requirement for a model column to be present in the data. This is not documented in `score()`.
I constructed some synthetic data and run into an error that was tricky to spot, caused by the fact that the sample column was automatically ignored: ``` library(dplyr) library(tidyr) library(scoringutils)...
This is a somewhat clunky proposal to add the ability to zoom in (and limit to 0) to plots with balooning predictive uncertainty, such as in https://covid19forecasthub.eu/reports/ensemble-report-2021-03-29.html
The following issues appear in `plot_pairwise_comparison`, mentioned in #217: > When type = "pval" in plot_pairwise_comparison the smaller_is_good argument did nothing. I have added documentation for this and added an...
Would it make sense to change plot_predictions so that it connects to day on which the forecast was made? This is the way it is done in the forecast hub...
Currently there is no way to have coverage displayed on the plot_score_table() plot. The following code will just remove it: ``` example_quantile %>% score() %>% add_coverage(by = c("model")) %>% summarise_scores(by...
Metadata
Owner
Metadata
Utilities for Scoring and Assessing Predictions