performance icon indicating copy to clipboard operation
performance copied to clipboard

Error in `model_performance.lavaan` when subsetting metrics for CFA - Inconsistency in documentation

Open jakoberr opened this issue 2 years ago • 0 comments

Find reprex below

https://github.com/easystats/performance/blob/0aa4a036ea87064afe4726ae5c2cc26e373c70d2/R/model_performance.lavaan.R#L115

I found an inconsistency between the documentation and the code for model_performance, using it to assess the performance of {lavaan} CFA models. This leads to an error, when a user is trying to subset the reported metrics according to the documentation.

The issue is, that the documentation for model_performance.lavaan gives this vector of possible metrics:

c("Chisq", "Chisq_DoF", "Chisq_p", "Baseline", "Baseline_DoF", "Baseline_p", "CFI", "TLI", "NNFI", "RFI", "NFI", "PNFI", "IFI", "RNI", "Loglikelihood", "AIC", "BIC", "BIC_adjusted", "RMSEA", "SMRM")

However, the metric names for a {lavaan} CFA model are different (e.g. Chi2 instead of Chisq). The names are specified correctly in the code, so I think the easiest option for fixing this, might be to update the documentation accordingly.

I just tested this for CFA models, I don't know, if this inconsistency also exists for other {lavaan} model types.

Reprex:

library(lavaan)
#> This is lavaan 0.6-8
#> lavaan is FREE software! Please report any bugs.
library(performance)

# Create a dummy cfa model to create performance measures

hz.data <- lavaan::HolzingerSwineford1939

hz.model <- '
  fct_1 =~ x1 + x2 + x3 + x4
  fct_2 =~ x5 + x6 + x7 + x8
'

hz.model.fit <- lavaan::cfa(hz.model, data = hz.data)

# No error if using metrics = "all"
performance::model_performance(hz.model.fit, metrics = "all")
#> # Indices of model performance
#> 
#> Chi2(19) | p (Chi2) | Baseline(28) | p (Baseline) |   GFI |  AGFI |   NFI |  NNFI |   CFI | RMSEA |    RMSEA  CI | p (RMSEA) |   RMR |  SRMR |   RFI |  PNFI |   IFI |   RNI | Loglikelihood |      AIC |      BIC | BIC_adjusted
#> ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
#> 207.212  |   < .001 |      789.827 |       < .001 | 0.853 | 0.721 | 0.738 | 0.636 | 0.753 | 0.181 | [0.16, 0.20] |    < .001 | 0.151 | 0.123 | 0.613 | 0.501 | 0.756 | 0.753 |     -3433.869 | 6901.738 | 6964.759 |     6910.844

# Error when trying to subset metrics like given in the documentation because
# the metric names in the documentation are wrong
performance::model_performance(hz.model.fit, metrics = c("Chisq"))
#> Error in `[.data.frame`(out, , metrics): nicht definierte Spalten gewählt

Created on 2022-07-01 by the reprex package (v2.0.0)

jakoberr avatar Jul 01 '22 05:07 jakoberr