Local ESS, quantile ESS, ESS change, and rank-based alternative to traceplot

[tjmahr]

Edit: Sorry @tjmahr, that I (Aki) hijack this issue.

Implement new plots as described in

• Aki Vehtari, Andrew Gelman, Daniel Simpson, Bob Carpenter, Paul-Christian Bürkner (2019): Rank-normalization, folding, and localization: An improved R-hat for assessing convergence of MCMC. arXiv preprint arXiv:1903.08008.

See also Dan's blog post

The reference code is available in monitorplot.R

Implement versions of functions

• [x] mcmc_rank_hist, mcmc_rank_overlay for rank plots
• [x] mcmc_ecdfd_r_scale for ECDF difference plots (see Fig. 14c in SBC paper)
• [ ] mcmc_ess_local for local efficiency plots
• [ ] mcmc_ess_quantile for quantile efficiency plots
• [ ] mcmc_ess_change for efficiency change plots

Feel free to come up with better function names, add checks, and it's fine to make separate PRs for these, if you want to implement just one.

For three last ones see also stan-dev/rstan#617

[avehtari]

@tjmahr's comments on rank plot

• If we take the input into mcmc_trace(), we know the chain id and values of the iterations. Then we can compute ranks with dplyr::row_number() and facet by chain id.
• Sanity check: Plot should be perfectly uniform when model has just one chain. Edit (Aki): The model should never have just one chain.

[avehtari]

@tjmahr, @paul-buerkner, @jgabry suggestions for the function names?

[paul-buerkner]

I think we could also use the mcmc_ prefix for the ESS plots, something like:

mcmc_ess_local
mcmc_ess_quantile
mcmc_ess_change

The rank plots could simply be named mcmc_rank or similar.

[jgabry]

I've been traveling and am just catching up on this. I like the function names suggested by @paul-buerkner

[jgabry]

Related to this issue, we'll also need to update the bayesplot MCMC diagnostics vignette.

[avehtari]

Added to todo list: mcmc_ecdfd_r_scale for ECDF difference plots (see Fig. 14c in SBC paper)

[avehtari]

example of mcmc_ecdfd_r_scale plot https://twitter.com/avehtari/status/1122940123522502657

[jgabry]

Thanks Aki

[ASKurz]

Would it be possible to add a facet_args argument to mcmc_rank_overlay()?

[jgabry]

Yeah I actually recently merged a PR from @hhau that added this (https://github.com/stan-dev/bayesplot/pull/221), so it should be available now if you install bayesplot from GitHub.

[ASKurz]

Excellent! Thanks for the heads up.

[timwaterhouse]

Hi all. It looks like there hasn't been any activity here for a while. Is there still interest in pursuing the remaining plot functions from this issue?

[avehtari]

ECDF difference plots were added by #282 ess-plots are still missing, but it would be nice to have them

[timwaterhouse]

Would a PR be welcome for those ESS plots that essentially collects the relevant code from @avehtari's reference code (along with tests etc)? Are there some complexities about bayesplot that I'd need to be aware of to make this work?

[avehtari]

Would a PR be welcome

Yes! You can make a PR, and Jonah can then provide feedback to take into account the bayesplot requirements