mvgam icon indicating copy to clipboard operation
mvgam copied to clipboard

Add support for `E_loo()`

Open nicholasjclark opened this issue 1 year ago • 1 comments

PSIS-LOO can be used to generated weighted expectations and predictions, valuable for asking what predictions we would get if high-leverage "outliers" were downweighted: https://paul-buerkner.github.io/brms/reference/loo_predict.brmsfit.html.

nicholasjclark avatar Apr 09 '24 03:04 nicholasjclark

Need to also update loo.mvgam to use posterior predictions rather than the draws generated from Stan. This is better for getting a less biased view of model fit, particularly for highly flexible AR / RW models:

loo_hcs = function(x, ...){
  x$series_names <- levels(x$obs_data$series)
  logliks <- logLik(x, 
                    linpreds = predict(x,
                                       newdata = x$obs_data,
                                       type = 'link',
                                       summary = FALSE,
                                       process_error = TRUE),
                    newdata = x$obs_data,
                    family_pars = mvgam:::extract_family_pars(x))
  logliks <- logliks[,!apply(logliks, 2, function(x) all(!is.finite(x)))]
  releffs <- loo::relative_eff(exp(logliks),
                               chain_id = sort(rep(1:x$model_output@sim$chains,
                                                   (NROW(logliks) /
                                                      x$model_output@sim$chains))))
  loo::loo(logliks, r_eff = releffs, ...)
}

nicholasjclark avatar May 07 '24 04:05 nicholasjclark