brms icon indicating copy to clipboard operation
brms copied to clipboard

add meaningful convergence statistics to summary.brmsfit() for class 'brmsfit_multiple'

Open wpetry opened this issue 6 years ago • 2 comments

The output of summary.brmsfit() for objects of class brmsfit_multiple reports an Rhat statistic that aggregates across chains fit to different imputed datasets, often giving rise to a false positive indication of non-convergence. Although this behaviour and the way to check dataset-level convergence is well-documented in a package vignette (ht: @paul-buerkner), it'd be nice to have some indication of these diagnostics in the summary output (and a more nuanced warning when some aggregate Rhats > 1.1).

Perhaps the least intrusive would be to add a max_Rhat column to the summary output? It would signal non-convergence clearly and succinctly with low likelihood of breaking changes for other functions (e.g. tidiers in broom and it's relatives).

wpetry avatar Jan 04 '19 15:01 wpetry

Good idea. I will have to think a little bit more of an appropriate way to display diagnostics, though. Likely, I will wait until the new convergence diagnostics for (R)stan are implemented.

paul-buerkner avatar Jan 09 '19 08:01 paul-buerkner

FWIW, I second that. For now, I check the convergence for brm_multiple model foo by requesting max(apply(foo$rhats, 1 max))

EmmanuelCharpentier avatar Jun 18 '19 13:06 EmmanuelCharpentier