Improvement of assumption checks section in ANOVA procedure

[vicenteinefo]

• Enhancement: To add assumption checks of normality of residuals (Shapiro will test)
• Purpose: To have an objective assessment of normality
• Use-case:

Additional context

[github-actions[bot]]

This issue will be automatically closed in 42 days due to inactivity. Feel free to leave a comment if you believe this is still relevant.

[vicenteinefo]

And what about add an option to Save a the residuals in a new computed variables in the dataset and then would be possible to apply the Shapiro will test for this new variable?

[tomtomme]

@JohnnyDoorn & @Kucharssim & @fqixiang Summing up:

Variance tests

• [ ] Breusch-Pagan's + White's p-value for testing H0: homoscedastic residuals in Regressions
• [ ] Levenes-Test of Median & Trimmed Mean for ANOVAs Textbook https://www.itl.nist.gov/div898/handbook/eda/section3/eda35a.htm In R: https://search.r-project.org/CRAN/refmans/lawstat/html/levene.test.html

Normality Tests

• [X] Kolmogorov-Smirnov within Distribution module
• [x] Shapiro-Wilk in Correlation & Descriptives & Manova & Distribution module
• [x] QQ-Plots within RM ANOVA & tTests & Regressions
• [x] 95% CIs for QQ-Plot in Distributions module
• [ ] Kolmogorov-Smirnov & shapiro-wilk within (RM) AN(C)OVA (& Regression, however there we can now export residuals to test in distribution module)
• [ ] QQ-Plot within MANOVA
• [ ] 95% CIs for QQ-Plot in Regressions & (RM) AN(C)OVAs

Linearity Test (logistic)

• [ ] assumption of "linearity in the logit" via Box & Tidwell, 1962 or Hosmer & Lemeshow, 1989 or Menard, 2002, 2010 or le Cessie-van Houwelingen-Copas-Hosmer

Box-Tidwell https://stackoverflow.com/questions/66618959/how-to-run-a-box-tidwell-test-in-r-to-test-for-a-linear-relationship-between-the https://statistics.laerd.com/spss-tutorials/binomial-logistic-regression-using-spss-statistics.php

Hosmer-lemeshow https://cran.r-project.org/web/packages/generalhoslem/index.html

le Cessie-van Houwelingen-Copas-Hosmer https://search.r-project.org/CRAN/refmans/DescTools/html/HosmerLemeshowTest.html https://www.rdocumentation.org/packages/rms/versions/6.8-0/topics/residuals.lrm

[JohnnyDoorn]

Great summing up @tomtomme - this is more or less in line with what I want to implement in the next months (QQ plots in more analyses, export residuals in anova/regression, see if AIC/BIC can easily be added there), now that I have some more time to deciate to JASP again.

[wpsilvestre]

@tomtomme since the #2695 wil be tracked here, what is the status of such an implementation? Is it feasible to add such a feature to ANOVA?

[tomtomme]

@wpsilvestre This is the status from my summing up above:

• Kolmogorov-Smirnov & shapiro-wilk within (RM) AN(C)OVA & Regression are currently missing, however there we can now - with 0.19beta - export residuals to test in distribution module, where SW and KS are available. This is not super-user-friendly but a step in the right direction. I cannot say if and when SW and KS will be available directly in those modules. But I would guess that it is feasible

[wpsilvestre]

@tomtomme , it would be great if this test is made available directly in ANOVA module with the homoscedasticity test and Q-Q plots. But I also understand that this takes time and effort. Thanks for the feedback.

[TarandeepKang]

To avoid duplication of #2726 ( now closed)

The most widely used test is the Hosmer-Lemshshow for logistic regression and other glms. But it is no longer necessarily considered the best. The le Cessie-van Houwelingen-Copas-Hosmer shows better performance in simumulations and is recommended by Frank Harrell in his RMS textbook.

The Hosmer-lemeshow is implemented in a few different versions for logistic regression and other glms here https://cran.r-project.org/web/packages/generalhoslem/index.html

And

le Cessie-van Houwelingen-Copas-Hosmer is available

https://search.r-project.org/CRAN/refmans/DescTools/html/HosmerLemeshowTest.html

or

https://www.rdocumentation.org/packages/rms/versions/6.8-0/topics/residuals.lrm

Hosmer, D. W., Hosmer, T., Le Cessie, S., & Lemeshow, S. (1997). A Comparison of Goodness-of-Fit Tests for the Logistic Regression Model. Statistics in Medicine, 16(9), 965–980. https://doi.org/10.1002/(SICI)1097-0258(19970515)16:9<965::AID-SIM509>3.0.CO;2-O Hosmer, D. W., & Lemesbow, S. (1980). Goodness of fit tests for the multiple logistic regression model. Communications in Statistics - Theory and Methods, 9(10), 1043–1069. https://doi.org/10.1080/03610928008827941 le Cessie, S., & van Houwelingen, J. C. (1991). A Goodness-of-Fit Test for Binary Regression Models, Based on Smoothing Methods. Biometrics, 47(4), 1267–1282. https://doi.org/10.2307/2532385 LEMESHOW, S., & HOSMER, D. W., JR. (1982). A REVIEW OF GOODNESS OF FIT STATISTICS FOR USE IN THE DEVELOPMENT OF LOGISTIC REGRESSION MODELS1. American Journal of Epidemiology, 115(1), 92–106. https://doi.org/10.1093/oxfordjournals.aje.a113284

[JohnnyDoorn]

@TarandeepKang thanks for the suggestion! So I guess the best one to add would be the Cessie-van Houwelingen-Copas-Hosmer? As for the tildwell test (adding log of the predictor and adding the interaction?), this seems better to leave manually specified. It might have to wait for the next JASP version though, since we are already in the testing phase for 0.19. I did recently add a plot to logistic regression to visualize the model's predictions - I am not entirely sure, but could this also be used to test the linearity of the logit? It only shows the model predictions and not the data, so I don't think so..

[TarandeepKang]

Hi @JohnnyDoorn, If I were you I would use the DescTools implementation, and offer both, maybe with the usual tick box? I don't think you can use this plot in that way. I've never seen it done that way, but that doesn't mean is not possible. I'm not a statistician like you, just an interested user. As for the Menard references that Thomas gives above, I confess I have never come across that test, so you may need to do some more research? I don't understand the question (if indeed you're asking one) about the Box-Tidwell test? Apologies if I'm being obtuse!

Best,

Tarandeep

[JohnnyDoorn]

HI @TarandeepKang No worries - I'm just curious what could be useful for users! I'll look into the tickboxes for the Cessie and Hosmer tests from Desctools, that seems easy enough.

[TarandeepKang]

Sorry, just one more thing! It is worth pointing out that the generalhoslem package also contains versions of this test that are suitable for ordinal and multinomial regression. Could you consider adding those to the other GLMs?.

[PerPalmgren]

Great if a test for linearity for the logistic module was implemented as it is one important assumption to be tested in logistic regression, The assumption requires there to be a linear relationship between the continuous independent variables and the logit transformation of the dependent variable. Now my students have to leave JASP and test this in SPSS (and where it is a bit cumbersome). Great if JASP could provide a more elegant way to test this assumption for logistic regression.

[EJWagenmakers]

As a temporary workaround, you can do this manually: use compute columns to create the logit of the dependent variable, as well as squared versions of the independent variables. Then use linear regression and assess whether the quadratic term(s) are worth including.

[EJWagenmakers]

But I agree it would be good to include that test explicitly. @vandenman we have it for correlation already, right? Maybe we should add it to regression and logistic regression too...

[vandenman]

We have it also for the t-tests, probably a good idea to add it here too.

[tomtomme]

maybe lets track all this now here: https://github.com/jasp-stats/jasp-issues/issues/2878 This thread is a bit confusing

[PerPalmgren]

For logistic regression in JASP will a possibilty to test the assumption of "linearity in the logit" by included? As the litterature points aut that one of the most important assumptions in logistic regression is the linear relationship between any continuous independent variables and the logit transformation of the dependent variable.

[tomtomme]

@PerPalmgren sure, it is included in the list above and at https://github.com/jasp-stats/jasp-issues/issues/2878