The dataset datasets::sleep should be analysed as a one-sample dataset

[jhmaindonald]

This issue arises in connecton witht the vignette bayestestR::bayes_factors (I've not checked other vignettes). The data in the dataset datasets::sleep is from 10 matched pairs, with ID referring to the individual, and group (confusingly) to the drug. The model should be specified as:

set.seed(123)
library(rstanarm)

model <- stan_glm(
    formula = extra[group==1]-extra[group==2] ~ 1,
    data = sleep,
    prior = normal(0, 3, autoscale = FALSE)
)

Then one gets:

My_first_BF <- bayestestR::bayesfactor_parameters(model, null = c(-1,1))
My_first_BF
## Bayes Factor (Null-Interval) 

## Parameter   |   BF
------------------
## (Intercept) | 2.84

## * Evidence Against The Null: [-1.000, 1.000]

The difference is even greater for a 0 point NULL. I get a Bayes factor that equals 14.86

[mattansb]

True - the current example doesn't account for possible dependency in the data. However, this is just a simple example used for illustrative purposes.... (Also I can argue that I have a prior that the random effects are 0 😋)

Feel free to make pull request with this correction if you think it's worth the effort 💪

[jhmaindonald]

The Pearson correlation is 0.8. John.

On 30/09/2021, at 20:40, Mattan S. Ben-Shachar @.***> wrote:

True - the current example doesn't account for possible dependency in the data. However, this is just a simple example used for illustrative purposes.... (Also I can argue that I have a prior that the random effects are 0 😋)

Feel free to make pull request with this correction if you think it's worth the effort 💪

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/easystats/bayestestR/issues/466#issuecomment-930947506, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJ3NKD4DWEJG2VRYLWVGILUEQH7RANCNFSM5FBT4URQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[bwiernik]

In my experience, R users rarely do simple dependent sample t tests like this. If this example is kept and changed to account for dependency, I would suggest specifying it as a fixed effects model with the person id as a factor covariate. Otherwise another dataset for two-group comparison would be better.

[mattansb]

I don't see how making the analysis in this example more "accurate" would benefit any potential readers*. That section of the vignette is about univariate Bayes factors, not about mixed models, or dependencies or the assumptions of iid. Such information is irrelevant to the point of the exercise...

_{(*) Or myself, who would have to edit and re-write the example.}

Does the "incorrectness" of the model specification somehow take away from how a variety of hypothesis tests on a single parameter can be done which Bayes factors? IMO no...

[mattansb]