performance
performance copied to clipboard
easystats
Possible bug: error is thrown saying check_model is not implemented, if model formula contains a ratio
Here is a reprex of a potential bug. Ignore the fact that this is probably not a good model!
In this example, I fit two models that return identical output. In one model, I create a ratio of two variables within the model formula. In the other, I create a new column in the data frame by dividing the two columns, and then use that new name in the model formula. Oddly, check_model() throws a strange error in the first case: Error: check_model()not implemented for models of classlmerMod yet. This doesn't seem like the desired behavior. Thanks in advance for looking into this!
reprex
library(lme4)
library(easystats)
model1 <- lmer(incidence/size ~ period + (1|herd), data = cbpp)
cbpp$proportion <- cbpp$incidence / cbpp$size
model2 <- lmer(proportion ~ period + (1|herd), data = cbpp)
all.equal(coefficients(model1), coefficients(model2)) # TRUE: they are the same
check_model(model1) # Error: `check_model()` not implemented for models of class `lmerMod` yet.
check_model(model2) # Works
session info
I just updated all easystats packages before running this, and verified the bug is still present using the latest CRAN versions of easystats packages.
R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)
Matrix products: default
locale:
[1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8 LC_MONETARY=English_United States.utf8
[4] LC_NUMERIC=C LC_TIME=English_United States.utf8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] report_0.5.7 see_0.7.5 correlation_0.8.4 modelbased_0.8.6 effectsize_0.8.3 parameters_0.21.1 performance_0.10.3
[8] bayestestR_0.13.1 datawizard_0.7.1 insight_0.19.2 easystats_0.5.2 lme4_1.1-31 Matrix_1.5-1
loaded via a namespace (and not attached):
[1] httr_1.4.4 splines_4.2.2 foreach_1.5.2 qqconf_1.3.1 assertthat_0.2.1
[6] memuse_4.2-3 robustbase_0.95-1 ggrepel_0.9.1 pillar_1.9.0 lattice_0.20-45
[11] glue_1.6.2 minqa_1.2.5 colorspace_2.1-0 sandwich_3.0-2 pkgconfig_2.0.3
[16] twosamples_2.0.0 xtable_1.8-4 mvtnorm_1.1-3 patchwork_1.1.2 scales_1.2.1
[21] pracma_2.4.2 emmeans_1.8.2-090002 tibble_3.2.1 mgcv_1.8-41 generics_0.1.3
[26] farver_2.1.1 ggplot2_3.4.2 TH.data_1.1-1 withr_2.5.0 opdisDownsampling_0.8.2
[31] cli_3.6.1 survival_3.4-0 magrittr_2.0.3 estimability_1.4.1 fansi_1.0.4
[36] doParallel_1.0.17 nlme_3.1-160 MASS_7.3-58.1 benchmarkme_1.0.8 tools_4.2.2
[41] lifecycle_1.0.3 multcomp_1.4-20 munsell_0.5.0 qqplotr_0.0.6 compiler_4.2.2
[46] caTools_1.18.2 rlang_1.1.1 grid_4.2.2 nloptr_2.0.3 iterators_1.0.14
[51] rstudioapi_0.14 bitops_1.0-7 labeling_0.4.2 boot_1.3-28 gtable_0.3.3
[56] codetools_0.2-18 DBI_1.1.3 benchmarkmeData_1.0.4 R6_2.5.1 zoo_1.8-11
[61] dplyr_1.0.10 utf8_1.2.3 parallel_4.2.2 Rcpp_1.0.10 vctrs_0.6.2
[66] DEoptimR_1.0-12 tidyselect_1.2.0 coda_0.19-4
This issue seems to be resolved in one of our dev-versions, at least I cannot reproduce this issue:
library(lme4)
#> Loading required package: Matrix
library(performance)
model1 <- lmer(incidence / size ~ period + (1 | herd), data = cbpp)
check_model(model1)
#> Not enough model terms in the conditional part of the model to check for
#> multicollinearity.
Created on 2023-06-02 with reprex v2.0.2
Can you run easystats::install_latest() and see if the error persists?
Unfortunately the error persists for me after running install_latest().
Looking for newer package versions...
Installing following packages:
Package | Latest | Installed
----------------------------------
datawizard | 0.7.1.9 | 0.7.1
bayestestR | 0.13.1.1 | 0.13.1
performance | 0.10.4 | 0.10.3
effectsize | 0.8.3.11 | 0.8.3
modelbased | 0.8.6.3 | 0.8.6
see | 0.7.5.9 | 0.7.5
report | 0.5.7.4 | 0.5.7
I ran the code exactly as in your post and got the same error that check_model() is not yet implemented for models of that class. Thanks for looking into this!
Can you try again and include your session info?
library(lme4)
#> Loading required package: Matrix
library(performance)
model1 <- lmer(incidence / size ~ period + (1 | herd), data = cbpp)
check_model(model1)
#> Not enough model terms in the conditional part of the model to check for
#> multicollinearity.
sessioninfo::session_info()
#> ─ Session info ───────────────────────────────────────────────────────────────
#> setting value
#> version R version 4.3.0 (2023-04-21 ucrt)
#> os Windows 10 x64 (build 19045)
#> system x86_64, mingw32
#> ui RTerm
#> language (EN)
#> collate German_Germany.utf8
#> ctype German_Germany.utf8
#> tz Europe/Berlin
#> date 2023-06-02
#> pandoc 3.1.1 @ C:/Users/mail/AppData/Local/Pandoc/ (via rmarkdown)
#>
#> ─ Packages ───────────────────────────────────────────────────────────────────
#> package * version date (UTC) lib source
#> bayestestR 0.13.1.1 2023-06-01 [1] https://easystats.r-universe.dev (R 4.3.0)
#> benchmarkme 1.0.8 2022-06-12 [1] CRAN (R 4.3.0)
#> benchmarkmeData 1.0.4 2020-04-23 [1] CRAN (R 4.3.0)
#> bitops 1.0-7 2021-04-24 [1] CRAN (R 4.3.0)
#> boot 1.3-28.1 2022-11-22 [2] CRAN (R 4.3.0)
#> caTools 1.18.2 2021-03-28 [1] CRAN (R 4.3.0)
#> cli 3.6.1 2023-03-23 [1] CRAN (R 4.3.0)
#> codetools 0.2-19 2023-02-01 [2] CRAN (R 4.3.0)
#> colorspace 2.1-0 2023-01-23 [1] CRAN (R 4.3.0)
#> datawizard 0.7.1.9 2023-05-30 [1] https://easystats.r-universe.dev (R 4.3.0)
#> DEoptimR 1.0-13 2023-05-02 [1] CRAN (R 4.3.0)
#> digest 0.6.31 2022-12-11 [1] CRAN (R 4.3.0)
#> doParallel 1.0.17 2022-02-07 [1] CRAN (R 4.3.0)
#> dplyr 1.1.2 2023-04-20 [1] CRAN (R 4.3.0)
#> evaluate 0.21 2023-05-05 [1] CRAN (R 4.3.0)
#> fansi 1.0.4 2023-01-22 [1] CRAN (R 4.3.0)
#> farver 2.1.1 2022-07-06 [1] CRAN (R 4.3.0)
#> fastmap 1.1.1 2023-02-24 [1] CRAN (R 4.3.0)
#> foreach 1.5.2 2022-02-02 [1] CRAN (R 4.3.0)
#> fs 1.6.2 2023-04-25 [1] CRAN (R 4.3.0)
#> generics 0.1.3 2022-07-05 [1] CRAN (R 4.3.0)
#> ggplot2 3.4.2 2023-04-03 [1] CRAN (R 4.3.0)
#> ggrepel 0.9.3 2023-02-03 [1] CRAN (R 4.3.0)
#> glue 1.6.2 2022-02-24 [1] CRAN (R 4.3.0)
#> gtable 0.3.3 2023-03-21 [1] CRAN (R 4.3.0)
#> htmltools 0.5.5 2023-03-23 [1] CRAN (R 4.3.0)
#> httr 1.4.6 2023-05-08 [1] CRAN (R 4.3.0)
#> insight 0.19.2 2023-05-30 [1] https://easystats.r-universe.dev (R 4.3.0)
#> iterators 1.0.14 2022-02-05 [1] CRAN (R 4.3.0)
#> knitr 1.43 2023-05-25 [1] CRAN (R 4.3.0)
#> labeling 0.4.2 2020-10-20 [1] CRAN (R 4.3.0)
#> lattice 0.21-8 2023-04-05 [1] CRAN (R 4.3.0)
#> lifecycle 1.0.3 2022-10-07 [1] CRAN (R 4.3.0)
#> lme4 * 1.1-33 2023-04-25 [1] CRAN (R 4.3.0)
#> magrittr 2.0.3 2022-03-30 [1] CRAN (R 4.3.0)
#> MASS 7.3-60 2023-05-04 [1] CRAN (R 4.3.0)
#> Matrix * 1.5-4.1 2023-05-18 [1] CRAN (R 4.3.0)
#> memuse 4.2-3 2023-01-24 [1] CRAN (R 4.3.0)
#> mgcv 1.8-42 2023-03-02 [1] CRAN (R 4.3.0)
#> minqa 1.2.5 2022-10-19 [1] CRAN (R 4.3.0)
#> munsell 0.5.0 2018-06-12 [1] CRAN (R 4.3.0)
#> nlme 3.1-162 2023-01-31 [2] CRAN (R 4.3.0)
#> nloptr 2.0.3 2022-05-26 [1] CRAN (R 4.3.0)
#> opdisDownsampling 0.8.2 2022-05-24 [1] CRAN (R 4.3.0)
#> patchwork 1.1.2 2022-08-19 [1] CRAN (R 4.3.0)
#> performance * 0.10.4 2023-06-02 [1] local
#> pillar 1.9.0 2023-03-22 [1] CRAN (R 4.3.0)
#> pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 4.3.0)
#> pracma 2.4.2 2022-09-22 [1] CRAN (R 4.3.0)
#> purrr 1.0.1 2023-01-10 [1] CRAN (R 4.3.0)
#> qqconf 1.3.2 2023-04-14 [1] CRAN (R 4.3.0)
#> qqplotr 0.0.6 2023-01-25 [1] CRAN (R 4.3.0)
#> R.cache 0.16.0 2022-07-21 [1] CRAN (R 4.3.0)
#> R.methodsS3 1.8.2 2022-06-13 [1] CRAN (R 4.3.0)
#> R.oo 1.25.0 2022-06-12 [1] CRAN (R 4.3.0)
#> R.utils 2.12.2 2022-11-11 [1] CRAN (R 4.3.0)
#> R6 2.5.1 2021-08-19 [1] CRAN (R 4.3.0)
#> Rcpp 1.0.10 2023-01-22 [1] CRAN (R 4.3.0)
#> reprex 2.0.2 2022-08-17 [1] CRAN (R 4.3.0)
#> rlang 1.1.1 2023-04-28 [1] CRAN (R 4.3.0)
#> rmarkdown 2.21 2023-03-26 [1] CRAN (R 4.3.0)
#> robustbase 0.95-1 2023-03-29 [1] CRAN (R 4.3.0)
#> scales 1.2.1 2022-08-20 [1] CRAN (R 4.3.0)
#> see 0.7.5.9 2023-06-02 [1] local
#> sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.3.0)
#> styler 1.10.0 2023-05-24 [1] CRAN (R 4.3.0)
#> tibble 3.2.1 2023-03-20 [1] CRAN (R 4.3.0)
#> tidyselect 1.2.0 2022-10-10 [1] CRAN (R 4.3.0)
#> twosamples 2.0.0 2022-07-12 [1] CRAN (R 4.3.0)
#> utf8 1.2.3 2023-01-31 [1] CRAN (R 4.3.0)
#> vctrs 0.6.2 2023-04-19 [1] CRAN (R 4.3.0)
#> withr 2.5.0 2022-03-03 [1] CRAN (R 4.3.0)
#> xfun 0.39 2023-04-20 [1] CRAN (R 4.3.0)
#> yaml 2.3.7 2023-01-23 [1] CRAN (R 4.3.0)
#>
#> [1] C:/Users/mail/AppData/Local/R/win-library/4.3
#> [2] C:/Program Files/R/R-4.3.0/library
#>
#> ──────────────────────────────────────────────────────────────────────────────
Created on 2023-06-02 with reprex v2.0.2
> sessioninfo::session_info()
─ Session info ──────────────────────────────────────────────────────────────────────────────
setting value
version R version 4.2.2 (2022-10-31 ucrt)
os Windows 10 x64 (build 19044)
system x86_64, mingw32
ui RStudio
language (EN)
collate English_United States.utf8
ctype English_United States.utf8
tz America/New_York
date 2023-06-02
rstudio 2023.03.0+386 Cherry Blossom (desktop)
pandoc 2.18 @ C:\\Users\\qdread\\AppData\\Local\\Pandoc\\pandoc.exe
─ Packages ──────────────────────────────────────────────────────────────────────────────────
package * version date (UTC) lib source
bayestestR 0.13.1.1 2023-06-01 [1] https://easystats.r-universe.dev (R 4.2.3)
boot 1.3-28 2021-05-03 [2] CRAN (R 4.2.2)
cli 3.6.1 2023-03-23 [1] CRAN (R 4.2.3)
datawizard 0.7.1.9 2023-05-30 [1] https://easystats.r-universe.dev (R 4.2.3)
insight 0.19.2 2023-05-23 [1] CRAN (R 4.2.3)
lattice 0.20-45 2021-09-22 [2] CRAN (R 4.2.2)
lme4 * 1.1-31 2022-11-01 [1] CRAN (R 4.2.2)
MASS 7.3-58.1 2022-08-03 [2] CRAN (R 4.2.2)
Matrix * 1.5-1 2022-09-13 [2] CRAN (R 4.2.2)
minqa 1.2.5 2022-10-19 [1] CRAN (R 4.2.2)
nlme 3.1-160 2022-10-10 [2] CRAN (R 4.2.2)
nloptr 2.0.3 2022-05-26 [1] CRAN (R 4.2.2)
performance * 0.10.4 2023-06-02 [1] https://easystats.r-universe.dev (R 4.2.3)
Rcpp 1.0.10 2023-01-22 [1] CRAN (R 4.2.3)
rstudioapi 0.14 2022-08-22 [1] CRAN (R 4.2.2)
sessioninfo 1.2.2 2021-12-06 [1] CRAN (R 4.2.2)
[1] C:/Users/qdread/AppData/Local/R/win-library/4.2
[2] C:/Program Files/R/R-4.2.2/library
My student and I are having similar issues with check_model(). I have performance v. 0.10.2, and my student has v. 0.10.4. We are both using Macs and RStudio (2023.03.1+446 for me, 2023.06.0+421 for my student). My student updated R yesterday (and easystats today), but I am running R v. 4.2.2.
- I ran model1 from strengejacke's comment on 2023-06-02 and got Error:
check_model()not implemented for models of classlmerModyet. However, if I set the response variable to just incidence (rather than incidence / size) then check_model works(). - I have been running various models with lmer, glmer with family=poisson, and glmer.nb with no problem, with performance v. 0.10.2.
- My student set up a logistic regression (glmer, family=binomial). check_model() gives an error that it is not implemented for models of class glmerMod on her computer with performance v. 0.10.4, but it works fine on my computer with v. 0.10.2. (We are running the same script. It is on a server.)
- On my student's computer we got an error about check_model() not being implemented for glmerMod objects when we did a Poisson regression (glmer, family=poisson), and an error that check_model has not been implemented for lmerModLmerTest when we did a linear mixed model with lmer.
For future reference, "Error: check_model() not implemented for models of class XX yet" is the generic message when anything goes wrong with extracting information from the model (e.g. https://github.com/easystats/performance/issues/678). With development versions of packages, options(easystats_errors = TRUE) should return the original message and make debugging easier (see https://github.com/easystats/performance/issues/691) ...
Furthermore, since June 2023, we also added more informative error messages in various places. This still might not capture every possible error, but as Ben wrote, setting options(easystats_errors = TRUE) should return the original error message in all (for users) relevant occasions.
Fixed in #751
library(lme4)
#> Loading required package: Matrix
library(easystats)
#> # Attaching packages: easystats 0.7.2.3
#> ✔ bayestestR 0.13.2.5 ✔ correlation 0.8.5
#> ✔ datawizard 0.12.0 ✔ effectsize 0.8.9
#> ✔ insight 0.20.2.1 ✔ modelbased 0.8.8
#> ✔ performance 0.12.0.11 ✔ parameters 0.22.0.2
#> ✔ report 0.5.9 ✔ see 0.8.4.7
cbpp$proportion <- cbpp$incidence / cbpp$size
model1 <- lmer(incidence/size ~ period + (1|herd), data = cbpp)
model2 <- lmer(proportion ~ period + (1|herd), data = cbpp)
all.equal(coefficients(model1), coefficients(model2)) # TRUE: they are the same
#> [1] TRUE
check_model(model1)
check_model(model2)
Created on 2024-07-14 with reprex v2.1.1