jasp-stats
[Bug]: Confidence intervals in plot wrong for paired t-Test
JASP Version
0.95.4
What analysis are you seeing the problem on?
paired t-Test
Bug Description
Example from data lib t-Tests "moon and aggression"
The plot that is shown:
Expected Behaviour
The plot that should be shown (manually made with flexplot)
Steps to Reproduce
- open the moon and aggression data
- note the differing variance of the data as seen in the raincloud plot
- not that the "descriptive" plots CIs do not show the differing variance
- maybe the CIs are already computed for the mean-difference? But then the plot should show the mean-diff and not the two means separated
Which raincloud plot do you refer to @tomtomme? Because the one in paired t-test only shows a boxplot, which is not the same as the sd/se; if you create raincloud plots from the descriptives module and choose to include confidnece intervals for the means (instead of the default boxplot), then those se's are calculated in the same way as for the paired t-test, if an ID variable is specified. So I'm still a little confused which two plots you are expecting to be the same while they aren't - perhaps you could share the jasp file so I can check it better. As far as I know, flexplot also does not do repeated measures, but I don't know that analysis so well.
In general, the paired t-test is all about the difference scores, so also looks at the sd of the differences - I think that's what the se's in the descriptives plot are based on. So it's not the same as pooling in a between-subjects design (welch vs. student).
I agree, this is confusing. We should bring this in line with the raincloud plots
OK I discussed this with Johnny, who clarified this for me. Essentially, the kind of intervals that JASP shows are based on the statistical model concerning the differences in the scores. Loftus & Masson have advocated these intervals, with Morey contributing as well. So this is standard in the field. We will clarify this in the help file, point to the papers, and do a tooltip to clarify. I think the Loftus and Masson paper is this one: https://link.springer.com/article/10.3758/BF03210951. In other words: this is a feature, not a bug.
good to know. Inclusion in the help file would be cool! That way I can explain to the students, why ESCI does it differently.
I added the issue to the list of fixes for help files here https://github.com/jasp-stats/jasp-issues/issues/2529 hence closing