jasp-stats
[Feature Request]: Hedges g* with non-pooled SD in addition/instead of current implementation with Welch's independent t-test
Description
Improved effect size estimator
Purpose
No response
Use-case
Whenever conducting Welch's t-test. Which, incidentally Lakens and his colleagues think should be the default
Is your feature request related to a problem?
current estimator may not be optimal
Is your feature request related to a JASP module?
T-Tests
Describe the solution you would like
Ad this new estimator
Describe alternatives that you have considered
No response
Additional context
Delacre, M., Lakens, D., & Leys, C. (2017). Why Psychologists Should by Default Use Welch’s t-test Instead of Student’s t-test. International Review of Social Psychology, 30(1). https://doi.org/10.5334/irsp.82
Delacre, M., Lakens, D., Ley, C., Liu, L., & Leys, C. (2021, May 7). Why Hedges’ g*s based on the non-pooled standard deviation should be reported with Welch’s t-test. https://doi.org/10.31234/osf.io/tu6mp
Code: https://github.com/mdelacre/deffectsize
I am sure similar arguments can be made for reporting medians rather than means, and for reporting nonparametric tests instead of parametric tests. If users want the Welch, they can select it. On the other hand, even R reports the Welch by default. I am therefore torn. @JohnnyDoorn what do you think? As far as Hedges g* based on the non-pooled standard deviation is concerned, I am open to including this as an option. @JohnnyDoorn, @vandenman what do you think?
For what it's worth, I would say that making the Welch the default is a secondary issue. I do think that the Hedges G* would be a good addition. The fact that R uses Welch as the default is potentially a strong argument for conformity if nothing else. Although I suppose another question is would you prefer not to impose such a default as a point of design principle? Lastly, I don't know what the default in SPSS is, I don't have it installed. But does that factor into your thinking? Anyway, I look forward to hearing what the team thinks, and I hope you don't make a regular habit of working on Sunday evening! :-)
Hi @TarandeepKang ,
Thanks for your insightful suggestion! We could revamp the effect sizes in such a way that the calculation differs for the Welch and Student t-test, maybe including a footnote to clarify. I think it makes sense to use the nonpooled SD in effect sizes for the Welch test, since that is also the SD at the core of the test statistic (although I've always found it a bit anticlimactic that there is no difference if group sizes are equal).
I would also be in favor of having Welch by default (this is also what we now advocate in our new Field book), and maybe even in ANOVA (although there it only exists for one-way designs). SPSS simply provides output for both Welch and Student by default in these cases.
Kind regards, Johnny
@TarandeepKang this one semms like a duplicate: https://github.com/jasp-stats/jasp-issues/issues/156 correct?
Not entirely certain, but subtly different thing I think? You will probably want to run it past EJ?
@TarandeepKang on reading #156 more careful again - it is about the paired test and this one about the independent. So yeah, not the same :D