mlr3learners icon indicating copy to clipboard operation
mlr3learners copied to clipboard

RegrKm SE prediction for epistemic uncertainty

Open mb706 opened this issue 3 years ago • 1 comments

RegrKm SE prediction with nugget currently predicts the epistemic uncertainty (uncertainty of the mean prediction) + aleatoric uncertainty (nugget SE, uncertainty that the model sees as random error). E.g. sampling lots of points with SE 0.1 noise and fitting a GP through them gives us

image

Where the GP goes through the points, the epistemic uncertainty is relatively low (it is the mean of a large sample) but the aleatoric uncertainty has SE 0.1 (nugget estimate).

What instead would be interesting would be the epistemic uncertainty alone

image

This is just

sqrt(pmax(p$sd^2 - self$model@covariance@nugget, 0))

I suggest we introduce a hyperparameter that gives the option to predict this, could be interesting for MBO.

mb706 avatar Feb 01 '21 15:02 mb706

PR would be welcome.

mllg avatar Mar 06 '21 22:03 mllg