[Discussion] Reporting $full_result of t-test ordering (confidence interval placement)

[Kalif-Vaughn]

Question: Is the full_result of a t-test correct APA style?

The full result using apa_print yields this format:

• ∆M = −3.35, 95% CI [−6.78, 0.09], t(535.71) = −1.92, p = .056

I typically see this in papers:

• t(535.71) = −1.92, p = .056, 95% CI [−6.78, 0.09]

In other words, most of the time I see the confidence intervals at the end of the result, and no reporting of the mean difference score.

I checked the APA manual, and it does not explicitly state that confidence intervals must follow the p-value. But on p. 117 of the 6th edition manual, it shows an example where the confidence intervals do follow the p-value.

Does anyone have any thoughts on this? I love all the information of the full_result but maybe we should re-order it so that it goes:

• ∆M = −3.35, t(535.71) = −1.92, p = .056, 95% CI [−6.78, 0.09]

What do you guys think?

[crsh]

Hi there,

I don't have any particularly strong opinions on this matter. I feel like it makes sense to have the CI follow the point estimate to immediately give a sense of the precision of the estimation procedure. Moreover, I get the sense, that the 6th edition APA manual isn't consistent on this matter either. For example on page 89 there is an example that reads "F(3, 132) = 19.58, p < .001, est η² = .31, 95% Cl [.17, .43]". Moreover, I'm a little reluctant to make any changes before I've seen the 7th edition. ;)

Any thoughts?

[Kalif-Vaughn]

Yeah, the APA manual does not explicitly state any rules about this (that I can find). Maybe they will further clarify it in the 7th edition.

But I agree with your plan: Wait and see in the 7th edition before changing anything. I personally like the way you have arranged it, I just wanted to be sure we weren't breaking some rule somewhere. 😄

Thanks for responding and all your hard work!

[crsh]

Thanks again for raising this issue @Kalif-Vaughn (cc: @mariusbarth). I just revisited it with the new publication manual in hand, and it seems there still is no clear guidance on how to do this. I spent some time browsing the manual and only found a few examples (in sections 6.4 and 6.43), which show a tendency but are not entirely consistent:

F(2, 71) = 3.38, p < .04, ηp2 = .087

F(2, 177) = 6.30, p = .002, est ω2 = .07

t(177) = 3.51, p < .001, d = 0.65, 95% CI [0.35, 0.95]

R^2 = .12, F(1, 148) = 20.18, p < .001, 95% CI [.02, .22]

So except for the model comparison example the point estimates and confidence intervals follow the NHST results. I'm not thrilled to implement this in a way that varies the order by test, so I would propose to use one consistet order for all cases. I tend towards NHST followed by estimate and CI. For your example this would imply the following order:

• t(535.71) = −1.92, p = .056, ∆M = −3.35, 95% CI [−6.78, 0.09]

What do you think? Does anyone have an opinion on this?

[Kalif-Vaughn]

Hello again! It is awesome that 7th Edition changes are on your mind. 👍

I agree that it should be consistent across tests. I think your proposal is great:

t(535.71) = −1.92, p = .056, ∆M = −3.35, 95% CI [−6.78, 0.09]

In my readings in the literature, I usually encounter it that way (with point estimates and confidence intervals after the NHST results). So this makes sense to me, but let's see if anyone else has thoughts!

Thanks again for all your work on this package.

[mariusbarth]

I agree with @Kalif-Vaughn that this is the best solution:

t(535.71) = −1.92, p = .056, ∆M = −3.35, 95% CI [−6.78, 0.09]