scoringutils

scoringutils copied to clipboard

reconsider name or sign for log_score

10

Currently the score called `"log_score"` actually computes the negative log score, as is described in the documentation [here](https://epiforecasts.io/scoringutils/articles/metric-details.html). I'd suggest either: - renaming the score to `"negative_log_score"` if you feel...

elray1

Both scripts are outdated and currently broken.

nikosbosse

Allow users to specify the quantile in `as_forecast_point.forecast_quantile()?

1

Is there a universe where we want to allow people to specify this quantile? If yes I think it can be a new issue _Originally posted by @seabbs in https://github.com/epiforecasts/scoringutils/pull/876#discussion_r1696751626_

nikosbosse

Add a reference for CRPS decomposition approach.

1

Add a reference for this approach. For example some mean CRPS decomposition discussed here; https://arxiv.org/pdf/2311.14122 _Originally posted by @seabbs in https://github.com/epiforecasts/scoringutils/pull/854#discussion_r1692024460_

seabbs

Refactor CRPS and WIS decomposition

1

Both functions duplicate a lot of code when `separate_results = TRUE`. This could maybe be simplified

nikosbosse

Do we do this anywhere? Just wondering if I could e.g. be messing things up unnoticedly in #985 as long as the components sum up to the overall crps. Perhaps...

sbfnk

## Description This PR closes #976. As mentioned in the issue I used the opportunity to make under/overprediction calculation concise. ## Checklist - [X] My PR is based on a...

sbfnk

New CRAN minor release

2

I would like to make more use of the new s3 systems and for this I really need them to be on CRAN. What do we think needs to happen...

seabbs

minimum data.table

3

turns out this line in the new non-randomised PIT https://github.com/epiforecasts/scoringutils/blob/07610998f9ec35e32b4510e3bee7fd8681a0ade3/R/metrics-sample.R#L582 relies on a fairly cutting-edge data.table version (1.16.0, released 25 August 2024) that supports a vector for `default` We either...

sbfnk

We only have `default` methods, but it would probably be cleaner to use `.data.frame` in addition to that

nikosbosse