performance icon indicating copy to clipboard operation
performance copied to clipboard

Revising `check_model()`

Open strengejacke opened this issue 11 months ago • 2 comments

Fixes #697

TODO or to check

  • [ ] #654
  • [ ] #500
  • [ ] #464
  • [ ] #376
  • [ ] #274
  • [ ] detrend in Q-Q plots (https://github.com/easystats/performance/pull/643#issuecomment-1999548863)
  • [ ] plot() for check_overdispersion()

strengejacke avatar Mar 18 '24 10:03 strengejacke

@bwiernik and @mccarthy-m-g - I think the implementation works quite well now for the first methods. The plot() method works fine for the Q-Q plots (https://github.com/easystats/see/pull/329). Things we should consider:

  • We would have to update the plot for overdispersion/zero-inflation checks (see https://github.com/easystats/performance/pull/643#issuecomment-1999544659). This is still based on the classical residuals, not the simulated ones. I have played around with revising the current code, copying the function .diag_overdispersion into a new .new_diag_overdispersion (see https://github.com/easystats/performance/blob/ad60db15550c406e533ad037107c81d81a66590d/R/check_model_diagnostics.R#L296). This did not really work - do you have any ideas how we can have new plots for overdispersion/zero-inflation? I found these plots quite informative and would like to keep them, beside the Q-Q plot. See also discussion here.

  • check_zeroinflation() and check_overdispersion() now rely on simulate_residuals() for zero-inflated or negative binomial models etc., so only the really "simple" models that returned identical results to DHARMa-tests still use the old code. Only Poisson mixed models also use the "old" code, which still could be inaccurate (see #464) - do we want to use simulate_residuals() in general for mixed models when calling check_zeroinflation() and check_overdispersion()?

  • How do we want to deal with y axis limits when detrend = TRUE in Q-Q plots? (see https://github.com/easystats/performance/pull/643#issuecomment-1999548863)

  • A method for check_autocorrelation() is not yet implemented. Since the DHARMa tests require additional information, I'm not sure about how to best implement such methods.

strengejacke avatar Mar 18 '24 11:03 strengejacke

Codecov Report

Attention: Patch coverage is 12.50000% with 7 lines in your changes missing coverage. Please review.

Project coverage is 57.73%. Comparing base (6e1eb60) to head (2be53b8). Report is 3 commits behind head on main.

Files Patch % Lines
R/check_model_diagnostics.R 12.50% 7 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #698      +/-   ##
==========================================
+ Coverage   57.69%   57.73%   +0.03%     
==========================================
  Files          87       87              
  Lines        6444     6464      +20     
==========================================
+ Hits         3718     3732      +14     
- Misses       2726     2732       +6     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

codecov[bot] avatar Mar 18 '24 11:03 codecov[bot]