bayestestR icon indicating copy to clipboard operation
bayestestR copied to clipboard

implement p-to-bf

Open strengejacke opened this issue 2 years ago • 7 comments

data(iris)
model <- lm(Petal.Length ~ Sepal.Length + Species, data = iris)
bayestestR::p_to_bf(model)
#>           Parameter            p           BF
#> 1       (Intercept) 1.005180e-11 3.693268e-10
#> 2      Sepal.Length 1.121002e-28 4.118823e-27
#> 3 Speciesversicolor 9.645641e-67 3.544035e-65
#> 4  Speciesvirginica 4.917626e-71 1.806851e-69

Created on 2022-05-16 by the reprex package (v2.0.1)

strengejacke avatar May 16 '22 08:05 strengejacke

Fixed: Now returns BF10 (instead of BF01, which is what the formula in the paper give...)

This paper is essentially trying to approximate the BIC to BF approximation (relaying on how Wald tests are asymptotically LRTs)...

Here is a demo (using @bwiernik pipebind!)

library(bayestestR)
library(pipebind)
#> Warning: package 'pipebind' was built under R version 4.1.3

m0 <- lm(mpg ~ 1, mtcars)
m1 <- lm(mpg ~ am, mtcars)
m2 <- lm(mpg ~ factor(cyl), mtcars)

bic_to_bf(BIC(m1), denominator = BIC(m0))
#> [1] 222.005
m1 |> 
  parameters::p_value() |> 
  bind(x, x[2, "p"]) |> 
  p_to_bf(n_obs = nrow(mtcars))
#>              p       BF
#> 1 0.0002850207 206.7413

bic_to_bf(BIC(m2), denominator = BIC(m0))
#> [1] 45355714
anova(m2) |> 
  parameters::p_value() |> 
  bind(x, x[1, "p"]) |> 
  p_to_bf(n_obs = nrow(mtcars))
#>              p       BF
#> 1 4.978919e-09 11835011

Created on 2022-05-17 by the reprex package (v2.0.1)

mattansb avatar May 17 '22 10:05 mattansb

If we do decide to merge this PR, I hope we don't endorse this down the line by adding these to any outputs.

mattansb avatar May 17 '22 10:05 mattansb

If we do decide to merge this PR, I hope we don't endorse this down the line by adding these to any outputs.

yes i agree

DominiqueMakowski avatar May 17 '22 10:05 DominiqueMakowski

tbh i wouldn't call it "pseudo" it's a bit long as a name and you know... what is "pseudo" really, if all approximations are pseudo then we should pseudo a lot of stuff ^^

DominiqueMakowski avatar May 17 '22 11:05 DominiqueMakowski

My thinking is that this is much more of a heuristic approximation than other approximations. And it’s a bit long for a reason 😜

bwiernik avatar May 17 '22 11:05 bwiernik

If you read the paper they are using the p-value to approximate the SE to approximate the LRT to approximate for the BIC comparisons that can be used to produce BFs. That is a long chain, with many assumptions...

I think p_to_pseudo_bf() is a good compromise...

mattansb avatar May 17 '22 11:05 mattansb

okay i'm convinced

DominiqueMakowski avatar May 17 '22 11:05 DominiqueMakowski

Codecov Report

Merging #529 (9098b03) into main (d494d36) will decrease coverage by 0.32%. The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #529      +/-   ##
==========================================
- Coverage   52.94%   52.61%   -0.33%     
==========================================
  Files          64       65       +1     
  Lines        4879     4909      +30     
==========================================
  Hits         2583     2583              
- Misses       2296     2326      +30     
Impacted Files Coverage Δ
R/p_to_bf.R 0.00% <0.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

codecov-commenter avatar Sep 10 '22 07:09 codecov-commenter

we should have merged this while i was onboard with p_to_pseudo_bf, now after looking back I think p_to_bf and put the dicslaimer in the title + docstrings should be enough. I mean people will at least check the title of the function ^^ wadayathink?

DominiqueMakowski avatar Sep 10 '22 07:09 DominiqueMakowski

I mean....... No...

Screenshot_20220910-140229.png

mattansb avatar Sep 10 '22 11:09 mattansb

BF is pseudo, anyway, so p_to_bf() includes all necessary information?

strengejacke avatar Sep 10 '22 11:09 strengejacke

I'm open for both suggestion, i.e. renaming to p_to_pseudo_bf(), or keep it as it is. @DominiqueMakowski do you plan to merge this before submitting bayestestR? I think it's feature complete now.

strengejacke avatar Sep 13 '22 10:09 strengejacke

I vote pseudo 👏

mattansb avatar Sep 13 '22 11:09 mattansb

I vote for keeping it as is

DominiqueMakowski avatar Sep 13 '22 11:09 DominiqueMakowski

image

strengejacke avatar Sep 13 '22 12:09 strengejacke

@easystats/core-team we need some help here...

strengejacke avatar Sep 13 '22 12:09 strengejacke

Compromise, leave the name and change the output to say “pseudo-BF”?

bwiernik avatar Sep 13 '22 13:09 bwiernik

Compromise, leave the name and change the output to say “pseudo-BF”?

Yeah, that's what I already did in the print method, as table header :-)

strengejacke avatar Sep 13 '22 13:09 strengejacke

image

strengejacke avatar Sep 13 '22 13:09 strengejacke

Yeah, that's what I already did

Meme lord + forward thinker nice

DominiqueMakowski avatar Sep 13 '22 13:09 DominiqueMakowski

Now let's hope this humble offering pleases @mattansb 🙏

DominiqueMakowski avatar Sep 13 '22 13:09 DominiqueMakowski

I'm good with it

bwiernik avatar Sep 13 '22 14:09 bwiernik

I'll take it

mattansb avatar Sep 13 '22 14:09 mattansb