parameters icon indicating copy to clipboard operation
parameters copied to clipboard

Should `model_parameters(<a mixed model>)` use `ci_method = "S"/"KR"` by default?

Open mattansb opened this issue 1 year ago • 4 comments

The current default for mixed models is to use residual dfs, but these are counter conservative. Should we default (when possible) to use Satterthwaite or Kenward-Roger dfs instead?

mattansb avatar Jul 09 '24 08:07 mattansb

I'm not sure if this should be the default. Maybe Sattherthwaite, but K-R can be really slow on large data sets.

Unlike many (or every?) other software, model_parameters() exactly describe the approximation methods, so we're already very transparent, which kind of DF are used to compute the p-values, so I would also be fine when we change nothing. But I'm open to switching to SW (not K-R though).

strengejacke avatar Jul 20 '24 11:07 strengejacke

The performance of the various approximations varies so much by model type and features (eg number of clusters, cluster size heterogeneity) that it's not clear to me which if any is most reasonable to choose as a default.

Cf https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4458010/ vs https://link.springer.com/article/10.3758/s13428-016-0809-y

Perhaps changing to S df for gaussian models but sticking with residual or between-within for other families?

bwiernik avatar Jul 20 '24 13:07 bwiernik

Do have any support for dfs for none gaussian models?

Anyway, I support this:

Perhaps changing to S df for gaussian models

mattansb avatar Jul 21 '24 06:07 mattansb

We have as approximation methods

  • https://easystats.github.io/parameters/reference/p_value_betwithin.html
  • https://easystats.github.io/parameters/reference/p_value_ml1.html

strengejacke avatar Jul 21 '24 07:07 strengejacke