lavaanExtra icon indicating copy to clipboard operation
lavaanExtra copied to clipboard

nice_fit() index cutoffs

Open TDJorgensen opened this issue 9 months ago • 5 comments

For review: openjournals/joss-reviews#5701

When nice_fit(..., nice_table = TRUE), there is a bottom row of "ideal values", with a table footnote indicating they were "proposed by Schreiber (2017)". This is inaccurate in 2 ways:

  1. Schreiber (2017) did not propose any cutoffs, but rather cited the source of fit-index cutoffs as Hu & Bentler (1999), who actually suggested RMSEA < .06, whereas your table suggests both < .05 and a range extending to .08, which originate from different sources:
    • Browne & Cudeck (1992) proposed RMSEA < .05 indicates close fit, RMSEA < .08 indicates reasonable fit, and RMSEA > .10 indices poor fit
    • MacCallum, Browne, & Sugawara (1996) proposed RMSEA between .08–.10 indicates mediocre fit
  2. Schreiber (2017, p. 640) explicitly did not advocate the cutoff values in the table:

Many researchers grew up with the combination suggestion of SRMR less than or equal to .08 and CFI greater than or equal to .95 (Hu & Bentler, 1999). This has not worked out in simulations (Sivo et al., 2006).

If anything, Schreiber's very next paragraph "proposed" using SRMR < .10 as a cutoff.

I do not recommend including cutoffs in the table, as doing so would perpetuate their misuse. Fit indices are not test statistics, and their suggested cutoffs are not critical values associated with known Type I error rates. Numerous simulation studies have shown how poorly cutoffs perform in model selection, including one of my own. Instead of test statistics, fit indices were designed to be measures of effect size (practical significance), which complement the chi-squared test of statistical significance. The range of RMSEA interpretations above is more reminiscent of the range of small/medium/large effect sizes proposed by Cohen for use in power analyses, which are as arbitrary as alpha levels, but at least they better respect the idea that (mis)fit is a matter of magnitude, not nearly so simple as "perfect or imperfect."

If you insist on including cutoffs in the table, I recommend:

  • including multiple suggestions (e.g., in multiple rows), with multiple subscripts that accurately link each suggestion to its original proposal. For example, Bentler & Bonett (1980) proposed incremental fit indices > .90 were acceptable.
  • calling them "suggested cutoffs" instead of "ideal values".

TDJorgensen avatar Oct 05 '23 14:10 TDJorgensen