.p_adjust : should be applied after keep/drop

[bwiernik]

Currently in parameters:::.extract_parameters_generic and related functions, we apply the keep/drop filters as a last step. This means that p values are adjusted based on all of the parameters, rather than only the ones in the table. We should probably do the p value adjustment as a last step, after filtering.

[strengejacke]

But keep/drop only affects the printed table, not the estimation. So doesn't the p-value-adjustment still apply to the full table of parameters?

[DominiqueMakowski]

Yes keep/drop was meant as an output tidier

[bwiernik]

If I am using keep to remove the control variables and just have the focal parameters for my hypotheses, I would expect the p value/CI adjustments to be based on just those focal parameters. If something isn't a test in my test set, I wouldn't adjust my p values for it.

[strengejacke]

library(parameters)
model <- lm(mpg ~ wt + cyl + gear + hp, data = mtcars)
model_parameters(model, summary = TRUE, p_adjust = "bonferroni")
#> Parameter   | Coefficient |   SE |         95% CI | t(27) |      p
#> ------------------------------------------------------------------
#> (Intercept) |       36.69 | 5.97 | [24.44, 48.94] |  6.15 | < .001
#> wt          |       -3.02 | 0.85 | [-4.77, -1.28] | -3.55 | 0.007 
#> cyl         |       -0.81 | 0.66 | [-2.17,  0.55] | -1.23 | > .999
#> gear        |        0.36 | 1.00 | [-1.69,  2.41] |  0.36 | > .999
#> hp          |       -0.02 | 0.02 | [-0.05,  0.01] | -1.38 | 0.896 
#> 
#> Model: mpg ~ wt + cyl + gear + hp (32 Observations)
#> Residual standard deviation: 2.551 (df = 27)
#> R2: 0.844; adjusted R2: 0.821
#> p-value adjustment method: Bonferroni
#> 
#> Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
#>   using a Wald t-distribution approximation.
model_parameters(model, summary = TRUE, keep = c("wt", "hp"), p_adjust = "bonferroni")
#> The 'keep' argument has more than 1 element. Merging into following
#>   regular expression: '(wt|hp)'.
#> Parameter | Coefficient |   SE |         95% CI | t(27) |     p
#> ---------------------------------------------------------------
#> wt        |       -3.02 | 0.85 | [-4.77, -1.28] | -3.55 | 0.003
#> hp        |       -0.02 | 0.02 | [-0.05,  0.01] | -1.38 | 0.358
#> 
#> Model: mpg ~ wt + cyl + gear + hp (32 Observations)
#> Residual standard deviation: 2.551 (df = 27)
#> R2: 0.844; adjusted R2: 0.821
#> p-value adjustment method: Bonferroni
#> 
#> Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
#>   using a Wald t-distribution approximation.
model_parameters(model, summary = TRUE, keep = c("cyl", "gear"), p_adjust = "bonferroni")
#> The 'keep' argument has more than 1 element. Merging into following
#>   regular expression: '(cyl|gear)'.
#> Parameter | Coefficient |   SE |        95% CI | t(27) |      p
#> ---------------------------------------------------------------
#> cyl       |       -0.81 | 0.66 | [-2.17, 0.55] | -1.23 | 0.462 
#> gear      |        0.36 | 1.00 | [-1.69, 2.41] |  0.36 | > .999
#> 
#> Model: mpg ~ wt + cyl + gear + hp (32 Observations)
#> Residual standard deviation: 2.551 (df = 27)
#> R2: 0.844; adjusted R2: 0.821
#> p-value adjustment method: Bonferroni
#> 
#> Uncertainty intervals (equal-tailed) and p-values (two-tailed) computed
#>   using a Wald t-distribution approximation.

^{Created on 2022-08-15 by the reprex package (v2.0.1)}

[bwiernik]

Awesome thanks!