loo
loo copied to clipboard
stan-dev
loo R package for approximate leave-one-out cross-validation (LOO-CV) and Pareto smoothed importance sampling (PSIS)
Addresses #136. Adds functions to compute `crps` and `scrps` and their `loo` versions. Happy to hear feedback!
Hi! This PR adds functions to compute additional LOO utilities: mean absolute error, mean squared error, and root mean squared error for regression problems and accuracy and balanced accuracy for...
Continuous Ranked Probability Score could be useful alternative for log score, As it is comparing predictive distribution to the observed data, it's a bit more complicated to implement than point...
Add R2
It would be nice if the loo objects weren't hardcoded to use elpd. For a study I wanted to include rmse and R2 and had to build fake loo objects...
Addresses #186: @jgabry @avehtari Extended `gpdfit()` to return the marginal posterior density of k in addition to k-hat and sigma-hat. We can also create a separate function but this seemed...
The current phrasing contains the same information but is a bit harder to interpret: ``` Computed from 4000 by 262 log-likelihood matrix ``` ParetoSmooth.jl is more explicit about this, with...
Example output from https://discourse.mc-stan.org/t/understanding-loo-results-for-non-standard-contaminated-controls-model/25493 ``` Pareto k diagnostic values: Count Pct. Min. n_eff (-Inf, 0.5] (good) 717 73.2% 151 (0.5, 0.7] (ok) 133 13.6% 116 (0.7, 1] (bad) 74 7.6%...
The current output ``` Pareto k diagnostic values: Count Pct. Min. n_eff (-Inf, 0.5] (good) 717 73.2% 151 (0.5, 0.7] (ok) 133 13.6% 116 (0.7, 1] (bad) 74 7.6% 38...
Remember to replace the short term fix for issue #151 (in PR #155) by using `posterior::autocovariance()` once the posterior package is on CRAN. This will avoid having to maintain the...
Only sort the tail values of the array, rather than the whole thing.