modelbased
modelbased copied to clipboard
Published
20 hours ago •
easystats
easystats
`estimate_contrasts()` ignoring `adjust` argument
Hi and thank you for your work on easystats. I wanted to point out that the contrast estimation function appears to handle the adjust argument incorrectly.
Changing the adjustment method has no effect on the p-values. A comparison with emmeans also shows that the default method used by modelbased is actually "tukey" instead of "holm", despite what is written in the documentation and what is printed in the footer of the output table:
data("penguins", package = "palmerpenguins")
lm_fit <- lm(body_mass_g ~ species, data = penguins)
modelbased::estimate_contrasts(lm_fit)
#> Registered S3 methods overwritten by 'parameters':
#> method from
#> as.double.parameters_kurtosis datawizard
#> as.double.parameters_skewness datawizard
#> as.double.parameters_smoothness datawizard
#> as.numeric.parameters_kurtosis datawizard
#> as.numeric.parameters_skewness datawizard
#> as.numeric.parameters_smoothness datawizard
#> print.parameters_distribution datawizard
#> print.parameters_kurtosis datawizard
#> print.parameters_skewness datawizard
#> summary.parameters_kurtosis datawizard
#> summary.parameters_skewness datawizard
#> Registered S3 method overwritten by 'modelbased':
#> method from
#> print.visualisation_recipe datawizard
#> Marginal Contrasts Analysis
#>
#> Level1 | Level2 | Difference | 95% CI | SE | t(339) | p
#> -----------------------------------------------------------------------------------
#> Adelie | Chinstrap | -32.43 | [ -194.85, 130.00] | 67.51 | -0.48 | 0.881
#> Adelie | Gentoo | -1375.35 | [-1510.44, -1240.27] | 56.15 | -24.50 | < .001
#> Chinstrap | Gentoo | -1342.93 | [-1511.00, -1174.86] | 69.86 | -19.22 | < .001
#>
#> Marginal contrasts estimated for species
#> p-value adjustment method: Holm (1979)
modelbased::estimate_contrasts(lm_fit, adjust = "tukey")
#> Marginal Contrasts Analysis
#>
#> Level1 | Level2 | Difference | 95% CI | SE | t(339) | p
#> -----------------------------------------------------------------------------------
#> Adelie | Chinstrap | -32.43 | [ -191.35, 126.50] | 67.51 | -0.48 | 0.881
#> Adelie | Gentoo | -1375.35 | [-1507.53, -1243.18] | 56.15 | -24.50 | < .001
#> Chinstrap | Gentoo | -1342.93 | [-1507.38, -1178.48] | 69.86 | -19.22 | < .001
#>
#> Marginal contrasts estimated for species
#> p-value adjustment method: Tukey
modelbased::estimate_contrasts(lm_fit, adjust = "sidak")
#> Marginal Contrasts Analysis
#>
#> Level1 | Level2 | Difference | 95% CI | SE | t(339) | p
#> -----------------------------------------------------------------------------------
#> Adelie | Chinstrap | -32.43 | [ -194.43, 129.57] | 67.51 | -0.48 | 0.881
#> Adelie | Gentoo | -1375.35 | [-1510.09, -1240.62] | 56.15 | -24.50 | < .001
#> Chinstrap | Gentoo | -1342.93 | [-1510.56, -1175.30] | 69.86 | -19.22 | < .001
#>
#> Marginal contrasts estimated for species
#> p-value adjustment method: Sidak
emmeans::emmeans(lm_fit, pairwise ~ species, adjust = "holm")$contrast
#> contrast estimate SE df t.ratio p.value
#> Adelie - Chinstrap -32.4 67.5 339 -0.480 0.6313
#> Adelie - Gentoo -1375.4 56.1 339 -24.495 <.0001
#> Chinstrap - Gentoo -1342.9 69.9 339 -19.224 <.0001
#>
#> P value adjustment: holm method for 3 tests
emmeans::emmeans(lm_fit, pairwise ~ species, adjust = "tukey")$contrasts
#> contrast estimate SE df t.ratio p.value
#> Adelie - Chinstrap -32.4 67.5 339 -0.480 0.8807
#> Adelie - Gentoo -1375.4 56.1 339 -24.495 <.0001
#> Chinstrap - Gentoo -1342.9 69.9 339 -19.224 <.0001
#>
#> P value adjustment: tukey method for comparing a family of 3 estimates
emmeans::emmeans(lm_fit, pairwise ~ species, adjust = "sidak")$contrasts
#> contrast estimate SE df t.ratio p.value
#> Adelie - Chinstrap -32.4 67.5 339 -0.480 0.9499
#> Adelie - Gentoo -1375.4 56.1 339 -24.495 <.0001
#> Chinstrap - Gentoo -1342.9 69.9 339 -19.224 <.0001
#>
#> P value adjustment: sidak method for 3 tests
Created on 2021-09-13 by the reprex package (v2.0.0)
Session info
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.0.2 (2020-06-22)
#> os macOS High Sierra 10.13.6
#> system x86_64, darwin17.0
#> ui X11
#> language (EN)
#> collate en_US.UTF-8
#> ctype en_US.UTF-8
#> tz America/New_York
#> date 2021-09-13
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date lib source
#> backports 1.2.1 2020-12-09 [1] CRAN (R 4.0.2)
#> bayestestR 0.11.0 2021-09-03 [1] CRAN (R 4.0.2)
#> cli 3.0.1 2021-07-17 [1] CRAN (R 4.0.2)
#> coda 0.19-4 2020-09-30 [1] CRAN (R 4.0.2)
#> codetools 0.2-18 2020-11-04 [1] CRAN (R 4.0.2)
#> crayon 1.4.1 2021-02-08 [1] CRAN (R 4.0.2)
#> datawizard 0.2.0.1 2021-09-02 [1] CRAN (R 4.0.2)
#> digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.2)
#> effectsize 0.4.5 2021-05-25 [1] CRAN (R 4.0.2)
#> ellipsis 0.3.2 2021-04-29 [1] CRAN (R 4.0.2)
#> emmeans 1.6.1 2021-06-01 [1] CRAN (R 4.0.2)
#> estimability 1.3 2018-02-11 [1] CRAN (R 4.0.1)
#> evaluate 0.14 2019-05-28 [1] CRAN (R 4.0.0)
#> fansi 0.5.0 2021-05-25 [1] CRAN (R 4.0.2)
#> fs 1.5.0 2020-07-31 [1] CRAN (R 4.0.2)
#> glue 1.4.2 2020-08-27 [1] CRAN (R 4.0.2)
#> highr 0.9 2021-04-16 [1] CRAN (R 4.0.2)
#> htmltools 0.5.1.1 2021-01-22 [1] CRAN (R 4.0.2)
#> insight 0.14.4 2021-09-02 [1] CRAN (R 4.0.2)
#> knitr 1.33 2021-04-24 [1] CRAN (R 4.0.2)
#> lattice 0.20-44 2021-05-02 [1] CRAN (R 4.0.2)
#> lifecycle 1.0.0 2021-02-15 [1] CRAN (R 4.0.2)
#> magrittr 2.0.1 2020-11-17 [1] CRAN (R 4.0.2)
#> MASS 7.3-54 2021-05-03 [1] CRAN (R 4.0.2)
#> Matrix 1.3-4 2021-06-01 [1] CRAN (R 4.0.2)
#> modelbased 0.7.0 2021-06-06 [1] CRAN (R 4.0.2)
#> multcomp 1.4-17 2021-04-29 [1] CRAN (R 4.0.2)
#> mvtnorm 1.1-2 2021-06-07 [1] CRAN (R 4.0.2)
#> palmerpenguins 0.1.0 2020-07-23 [1] CRAN (R 4.0.2)
#> parameters 0.14.0 2021-05-29 [1] CRAN (R 4.0.2)
#> pillar 1.6.2 2021-07-29 [1] CRAN (R 4.0.2)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.0.0)
#> plyr 1.8.6 2020-03-03 [1] CRAN (R 4.0.0)
#> purrr 0.3.4 2020-04-17 [1] CRAN (R 4.0.0)
#> Rcpp 1.0.7 2021-07-07 [1] CRAN (R 4.0.2)
#> reprex 2.0.0 2021-04-02 [1] CRAN (R 4.0.2)
#> rlang 0.4.11 2021-04-30 [1] CRAN (R 4.0.2)
#> rmarkdown 2.10 2021-08-06 [1] CRAN (R 4.0.2)
#> rstudioapi 0.13 2020-11-12 [1] CRAN (R 4.0.2)
#> sandwich 3.0-1 2021-05-18 [1] CRAN (R 4.0.2)
#> sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 4.0.2)
#> stringi 1.7.3 2021-07-16 [1] CRAN (R 4.0.2)
#> stringr 1.4.0 2019-02-10 [1] CRAN (R 4.0.0)
#> styler 1.5.1 2021-07-13 [1] CRAN (R 4.0.2)
#> survival 3.2-11 2021-04-26 [1] CRAN (R 4.0.2)
#> TH.data 1.0-10 2019-01-21 [1] CRAN (R 4.0.2)
#> tibble 3.1.3 2021-07-23 [1] CRAN (R 4.0.2)
#> utf8 1.2.2 2021-07-24 [1] CRAN (R 4.0.2)
#> vctrs 0.3.8 2021-04-29 [1] CRAN (R 4.0.2)
#> withr 2.4.2 2021-04-18 [1] CRAN (R 4.0.2)
#> xfun 0.25 2021-08-06 [1] CRAN (R 4.0.2)
#> xtable 1.8-4 2019-04-21 [1] CRAN (R 4.0.0)
#> yaml 2.2.1 2020-02-01 [1] CRAN (R 4.0.0)
#> zoo 1.8-9 2021-03-09 [1] CRAN (R 4.0.2)
#>
#> [1] /Library/Frameworks/R.framework/Versions/4.0/Resources/library
Just wanted to add a +1. This has broken a load of my learnr tutorials (in that text no longer matches output). Here's another reproducible example:
pupluv_tib <- read.csv("https://www.discovr.rocks/csv/puppy_love.csv") |>
dplyr::mutate(
dose = forcats::as_factor(dose)
)
pupluv_lm <- lm(happiness ~ puppy_love + dose, data = pupluv_tib)
modelbased::estimate_contrasts(pupluv_lm, fixed = "puppy_love", adjust = "holm")
modelbased::estimate_contrasts(pupluv_lm, fixed = "puppy_love", adjust = "bonferroni")
