powerCurve not wrapping user supplied tests

[pitakakariki]

This works fine:

> powerSim(fm, test=function(.) 1, nsim=5)

But this doesn't:

> powerCurve(fm, test=function(.) 1, nsim=5)
Calculating power at 8 sample sizes along x
Error in attr(test, "text")(fit, sim) : attempt to apply non-function

[mattansb]

Hey - thanks for the great package!

I just came across this same issue.. Until fixed, one should make sure the input test function has some functions as the "text" attribute:

library(lme4)
library(emmeans)
library(simr)


data(md_12.1, package = "afex")

mod_rt <- lmer(rt ~ angle * noise + (angle + noise|id), data = md_12.1)

test_lin_trend <- function(m, ...) {
  rez <- emmeans(m, ~ angle, lmer.df = "asym") |>
    contrast(method = "poly") |>
    summary()
  
  rez$p.value[1]
}

powerCurve(mod_rt, test = test_lin_trend, 
           breaks = c(1, 10),
           nsim = 5, along = "id")
#> Calculating power at 2 sample sizes along id
#> Error in attr(test, "text")(fit, sim) : attempt to apply non-function

attr(test_lin_trend, "text") <- function(...) NULL

powerCurve(mod_rt, test = test_lin_trend, 
           breaks = c(1, 10),
           nsim = 5, along = "id")
#> Calculating power at 2 sample sizes along id
#> by number of levels in id:
#>       1:  0.00% ( 0.00, 52.18) - 6 rows
#>      10: 100.0% (47.82, 100.0) - 60 rows
#> 
#> Time elapsed: 0 h 0 m 2 s

[pitakakariki]

Thanks Matt.

[HaiyangJin]

Hi @pitakakariki and @mattansb, many thanks for making this great package and the excellent example.

I wonder what I should do to test the power for both rez$p.value[1] and rez$pvalue[2] with custom function at the same time?

[pitakakariki]

Sorry, that's not a feature that's available yet. It's a good idea so I will try to implement it at some point.

[HaiyangJin]

Thanks! Do you think it makes sense to perform the power analysis twice for the two effects of interests with the same random seeds (before the exciting new feature is available)?

[pitakakariki]

Seed shouldn't matter either way as long as nsim is big enough. But yes currently you just need to run two separate powerSims.

[HaiyangJin]

I would like to calculate the power of observing the significant results of the two tests at the same time. Different seeds should not work in this case.

I wonder if I could perform the power analysis twice but with the same seed, and then use the p-values for different effects (from different power analysis) to calculate whether the two effects are significant at the same time (for each iteration/simulation).