microbenchmark icon indicating copy to clipboard operation
microbenchmark copied to clipboard

Document or make it easier to use list arg

Open richierocks opened this issue 8 years ago • 0 comments

Suppose I want to test how a function scales with respect to the size of its input.

Here's an example function that scales rather badly.

f <- function(n) {
  A <- matrix(runif(n * n), nrow = n, ncol = n)
  b <- 1:n
  qr.solve(A, b)
}

I can benchmark it as follows

library(microbenchmark)
bench <- microbenchmark(
  f(8),
  f(32),
  f(128),
  f(512),
  times = 10
)
plot(bench, log = "y")

This is pretty clunky though. What I really want to write is something like this:

library(microbenchmark)
n <- 2 ^ seq(3, 9, 2)
n <- setNames(n, n)

bench2 <- microbenchmark(
  list = lapply(n, f),
  times = 10
)
plot(bench2, log = "y")

Unfortunately, it seems that the expressions are evaluated before the benchmarking begins. (In fairness, the documentation does say that unevaluated expressions are needed.)

After rather a lot of messing around, the best that I could come up with was this.

bench3 <- microbenchmark(
  list = lapply(n, function(i) call("f", i)),
  times = 10
)
plot(bench3, log = "y")

This works, but it is pretty ugly, and was hard to find.

It would be useful if you could have a think about ways to make it easier to use the list argument to microbenchmark.

If you have a simpler way to use it, please add an example to example(microbenchmark) (or possibly even to a vignette).

Using some sort of lazy evaluation of the list elements so that lapply(n, f) works would be even better.

richierocks avatar May 02 '17 21:05 richierocks