TOSTER icon indicating copy to clipboard operation
TOSTER copied to clipboard

Unclear why `power_twoprop()` gives different sample size than seems to be required

Open wviechtb opened this issue 11 months ago • 2 comments

I was interested in using power_twoprop() for a sample size calculation for an equivalence test. Say the true proportions are .28 and .33, my margin of error is .10, and I want 80% power with alpha=.05. With:

TOSTER::power_twoprop(
   p1    = .28,
   p2    = .33,
   null  = .10,
   alpha = .05,
   power = .80,
   alternative = "equivalence"
)

the required sample size is calculated to be 1447.962. This did not match the value (n=1048) in Table 2 in Walker and Nowacki (2011) (https://doi.org/10.1007/s11606-010-1513-8), which was calculated based on the commercial PASS software package (which costs a measly $2,395 for a perpetual license -- academic pricing!).

So I did a quick simulation to see how often the bounds of the 90% CI fall within (-margin, margin):

simpower <- function(p1, p2, n, margin, alpha=.05) {
   iters <- 10000
   sig <- rep(NA, iters)
   for (i in 1:iters) {
      x1     <- rbinom(n, 1, p1)
      x2     <- rbinom(n, 1, p2)
      pr1    <- mean(x1)
      pr2    <- mean(x2)
      se     <- sqrt(pr1*(1-pr1)/n+pr2*(1-pr2)/n)
      crit   <- qnorm(1-alpha, lower.tail=TRUE)
      ci.lb  <- (pr1-pr2) - crit * se
      ci.ub  <- (pr1-pr2) + crit * se
      sig[i] <- ci.lb > -margin && ci.ub < margin
   }
   round(mean(sig), 2)
}

simpower(p1=.28, p2=.33, n=1448, margin=.10)

This yields a power of around 91%, so quite a bit above the requested 80%. When running this with n=1048:

simpower(p1=.28, p2=.33, n=1048, margin=.10)

then we get indeed 80% power. So something is off here.

It seems that power_twoprop() is based on Chow et al. (2008), which is also cited for example here:

http://powerandsamplesize.com/Calculators/Compare-2-Proportions/2-Sample-Equivalence

which does give the same results as power_twoprop(). But the simulation shows that this is not giving the power of the TOST procedure as far as I can tell. Any insights into what the issue is?

wviechtb avatar Sep 19 '23 09:09 wviechtb