Modify Rhat to compute rank-normalised rhat

[DavAug]

This issue partially addresses #1063 .

I'd like to first address rank-normalising rhat, because this just involves

• [ ] Computing the rank statistic of the pooled samples
• [ ] Translating the rank statistic to Normal quantiles

The existing rhat computation using the within and between chain variance is then done using the above constructed normal RV, see https://arxiv.org/abs/1903.08008 for more info.

For chains with finite mean and variance this transformation is supposed to leave the Rhat statistic invariant. The real benefit comes for heavy tailed distributions.

I would nevertheless argue for adding a flag to the rhat computation:

• [ ] rank_normalised=True which can be turned off, and effectively lead to skipping of the above mentioned transformation.

[ben18785]

Sounds good.

One thing I wonder is whether these sorts of diagnostics might actually be best handled by using an existing package. Ths Arviz package is quite well established and seems to offer a lot of what we want: https://arviz-devs.github.io/arviz/_modules/arviz/stats/diagnostics.html

Might save us reinventing the wheel?

On Sun, Jan 31, 2021 at 2:15 PM David Augustin [email protected] wrote:

This issue partially addresses #1063 https://github.com/pints-team/pints/issues/1063 .

I'd like to first address rank-normalising rhat, because this just involves

• Computing the rank statistic of each subchain
• Translate rank statistic to Normal quantiles

The existing rhat computation using the within and between chain variance is then done using the above constructed normal RV, see https://arxiv.org/abs/1903.08008 for more info.

For chains with finite mean and variance this transformation is supposed to leave the Rhat statistic invariant. The real benefit comes for heavy tailed distributions.

I would nevertheless argue for adding a flag to the rhat computation:

• rank_normalised=True which can be turned off, which will skip the above mentioned transformation.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/pints-team/pints/issues/1273, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCILKGCIY65KS533NTOLVDS4VQWJANCNFSM4W3MOQ3Q .

[DavAug]

Ahh cool stuff! In that case, we might as well not have these diagnostics in pints and refer to arviz?

[ben18785]

Yeah might be easier. Shall we discuss on Tuesday?

On Sun, Jan 31, 2021 at 3:04 PM David Augustin [email protected] wrote:

Ahh cool stuff! In that case, we might as well not have these diagnostics in pints and refer to arviz?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/pints-team/pints/issues/1273#issuecomment-770396094, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCILKHL6SNS3RJ2D223VILS4VWPHANCNFSM4W3MOQ3Q .