bayesplot icon indicating copy to clipboard operation
bayesplot copied to clipboard

Add warning about using ppc_stat stat="mean"

Open avehtari opened this issue 1 year ago • 0 comments

Most of the models do include parameters that do make the posterior predictive mean to match the observed mean, and the test statistic "mean" is perfectly dependent and not ancillary as required so that ppc would make some sense. stat="mean" might make sense given grouping if the groups don't have their own intercept parameters. I have recently seen too many times people posting ppc_stat plots with stat="mean" and the plots have zero information content. I guess we can't change the default to not break existing code.

  • When calling ppc_stat() with the default stat or with explicit stat="mean", create a warning saying "Test statistic "mean" is unlikely to be ancillary and thus likely to not provide any useful information. We recommend using some other test statistic." or if we want to avoid the word "ancillary" then warn: "Test statistic "mean" is very weak to detect anything interesting in most cases. We recommend using some other test statistic."
  • Change the ppc_stat doc to describe the problem with "mean" test statistic and use some other test statistic in the example

avehtari avatar Mar 15 '24 13:03 avehtari