Add non-Bayesian documentation

[mattansb]

Since bayestestR can really be used with any distribution of data, I think it would be good to have a vignette about using it with non-Bayeisan distributions - for example how to use bayestestR with bootstrapped estimates (as done in parameters). Should include:

• Central (median, mean, map)
• ETI (aka percentile interval)
• (Maybe add a BCa method fo CI?)
• equivalence_test

I would also like to explain why bootstrap estimates are not Bayesian (idea after just now getting an email asking how to compute a Bayes factor from a bootstrapped distribution). Also explain why some other posterior-probability based methods are not appropriate for non-Bayesian distributions (use Moreys CI paper about post-data inference).

[DominiqueMakowski]

Agreed, that'd be super useful

[strengejacke]

I would also like to explain why bootstrap estimates are not Bayesian

I think one difficulty to explain differences between Bayesian and Freq. is:

• Bayes does sampling, bootstrap is sampling. So why isn't bootstrapping "similar" to Bayesian approach?
• With uniform priors, but - when enough data - even with more informative priors, estimates are comparable. What's the difference?

I know there are many other advantages, in particular when you have mixed models (no convergence issues, "proper" statistics (like variance etc.) for random effects and so on, but indeed, for people not used to Bayesian approach, they don't see the "magic" if results are very alike.

[DominiqueMakowski]

I would also like to explain why bootstrap estimates are not Bayesian

I think that what easystats users expect here, rather some lengthy theoretical rant, is a practical perspective. For instance, mention that the main difference is the presence of priors, but also say that the more uninformative and flat priors you have, the closer you get from maximum likelihood (i.e., ~frequentism). It could almost be seen as a continuum (I reckon some won't like this 😁).

I know there are many other advantages

And then yeah, we could present it like a "pros and cons" list:

• No convergence issues for complex models
• True and straightforward statistics
• Less heavy on assumptions (?)
• Natural way of regularizing
• But slow
• ...

We also need to show somewhere at the beginning how a user can easily get a Bayesian or a Bootstrapped results using easystats.

[mattansb]

I think that what easystats users expect here, rather some lengthy theoretical rant, is a practical perspective.

You know me too well... But I must rant at least a little bit! 😅

I think that this doc can be split into:

1. Working with bootstrapped distributions in bayestestR
2. Why this is not a Bayesian method.
   • But when will they still give similar inferential results.
3. The advantages of Bayes over bootstrap.

[mattansb]

<rant> I agree that results are often similar numerically, but I also think we shouldn't only strive to make analyses easy, but also push for the correct interpretation. I sometimes fear that by making analyses too easy, novice users will miss-use or miss-interpret our fantastic tools. I don't think we shouldn't lower our expectations from users (or from their ability to understand stats) just because they're probably only doing very basic analysis, and that any miss-attributions of statistic properties will probably be overall negligible (cf. why pd is not the p-value, why confidence intervals aren't credible intervals, even though r > 0.99 between them)... Being a popular and low-friction solution has its disadvantages - we have a responsibility! </rant>

[mattansb]

I swear I am generally very sane person!

[bwiernik]

I mean, Efron has done quite a lot of work over the decades on both Bayesian and frequentist foundations of bootstrap distributions, so I don't think it's necessary to say "bootstrap isn't Bayesian"

[bwiernik]

With respect to the question of Bayes Factor for bootstrap distributions, a reasonable approach could be to compute a relative likelihood ( exp( ∆AIC / 2 ) ) using an empirical or bootstrap quasilikelihood (cf. https://www.sciencedirect.com/science/article/abs/pii/S0167947307002587).

It would potentially be generally interesting to provide equivalent functionality relying on posterior, bootstrap, likelihood, or confidence distributions/distribution functions.

[bwiernik]

In addition to bootstrap distributions, I think it would be nice to provide an option for analytic distributions, eg to present a scaled shifted t distribution for regression model parameters.