Document or make it easier to use list arg

[joshuaulrich]

This was reported as https://github.com/olafmersmann/microbenchmark/issues/6.

---

Suppose I want to test how a function scales with respect to the size of its input.

Here's an example function that scales rather badly.

f <- function(n) {
  A <- matrix(runif(n * n), nrow = n, ncol = n)
  b <- 1:nf
  qr.solve(A, b)
}

I can benchmark it as follows

library(microbenchmark)
bench <- microbenchmark(
  f(8),
  f(32),
  f(128),
  f(512),
  times = 10
)
plot(bench, log = "y")

This is pretty clunky though. What I really want to write is something like this:

library(microbenchmark)
n <- 2 ^ seq(3, 9, 2)
n <- setNames(n, n)

bench2 <- microbenchmark(
  list = lapply(n, f),
  times = 10
)
plot(bench2, log = "y")

Unfortunately, it seems that the expressions are evaluated before the benchmarking begins. (In fairness, the documentation does say that unevaluated expressions are needed.)

After rather a lot of messing around, the best that I could come up with was this.

bench3 <- microbenchmark(
  list = lapply(n, function(i) call("f", i)),
  times = 10
)
plot(bench3, log = "y")

This works, but it is pretty ugly, and was hard to find.

It would be useful if you could have a think about ways to make it easier to use the list argument to microbenchmark.

If you have a simpler way to use it, please add an example to example(microbenchmark) (or possibly even to a vignette).

Using some sort of lazy evaluation of the list elements so that lapply(n, f) works would be even better.

[HughParsonage]

Is this suitable?

microbenchmark_by <- function(X, FUN, times = 100, unit, check = NULL, control = list()) {
  fx <- match.fun(FUN)
  newlist <- lapply(X, function(i) bquote(fx(.(i))))
  fx.char <- deparse(substitute(FUN))
  names(newlist) <- vapply(X, function(x) paste0(fx.char, "(", x, ")"), character(1L))
  if (missing(unit)) {
    microbenchmark(list = newlist, times = times, check = check, control = control)
  } else {
    microbenchmark(list = newlist, times = times, unit = unit, check = check, control = control)
  }
}

> microbenchmark_by(1:10, f)
Unit: microseconds
  expr        min          lq        mean      median          uq        max neval cld
  f(1)     71.078     80.1135    107.8153     86.1360    129.2050    267.144   100 a  
  f(2)     73.186     80.1130    140.1798     85.2335     94.8710   4352.903   100 a  
  f(3)     81.016     88.8470    120.6243     96.3765    135.8310    272.565   100 a  
  f(4)    101.497    108.4240    128.7591    113.0920    143.3605    212.932   100 a  
  f(5)    170.466    180.1035    202.1498    186.4280    205.7040    433.694   100 a  
  f(6)    449.957    461.5530    592.9683    470.8900    519.8310   5581.401   100 a  
  f(7)   1613.402   1638.2490   1878.3291   1697.5810   1789.4395   9040.413   100 a  
  f(8)   7515.256   7963.1045   8674.9203   8092.1590   8273.3170  16441.222   100 a  
  f(9)  43461.867  44107.5885  57870.3455  44711.4475  48893.4325 219280.833   100  b 
 f(10) 260278.474 271003.6675 326673.2987 279675.5910 428058.2885 465327.215   100   c

[joshuaulrich]

@richierocks, I know it's been awhile since you first opened this issue under Olaf's account, but I would be interested in your thoughts on Hugh's suggestion.