mlr3proba
mlr3proba copied to clipboard
mlr-org
Interval and confidence set output types
The following output types may be useful to have in the probabilistic regression task:
- quantile-based intervals, e.g., "95% CI"
- scale/location intervals, i.e., conditional, combined mean-variance predictions
- confidence sets at a given confidence percentage, for discrete predictive distributions over the reals (e.g., counts)
This is a more general question, namely is "interval probabilistic regression" suitably different enough to be a different task. If not then how many predict types are available for probabilistic regression? At the very least distr, but then interval? qqinterval ?
Hm, eventually, I think these are all just predict types of "regression" - similar how deterministic and probabilistic are predict types of classification.
In the interim, the interval return types could be predict types of the "not in mlr3base" regression.
Though, there are at least two important kinds of predictive intervals: quantile based (with an alpha/confidence parameter), and mean-variance predictions.
Can we argue mean-variance is returned in mlr3 base though because several learners return response and se?
actually: yes.
Though there aren't any losses implemented that evaluate the "response/se" pair, are there?
Similarly, you could also design quantile-based interval predictions as return types being "the x-th quantile", rather than making intervals a return object in their own right.
Yes this is what I was thinking, that whilst this is possible in mlr3 there is nothing one can do with it (i.e. evaluate). So it isn't clear to me if a) mlr3proba should add interface points between mlr3 and measures that incorporate both response and se, b) mlr3proba should re-implement these, c) we add measures to mlr3measures and point users to these for use with mlr3 and response/se returns
Though, there are special methods that can predict multiple quantiles at the same time, especially symmetric pairs of alpha-% and (1-alpha)-% quantiles.
I guess the main question is: is there an easy way to query both alpha-% and (1-alpha)-% quantiles as a return type of a regression method within the current interface design. Where alpha is set by the user.