Feature request: relative time for each parameter with bench::press

[MatthieuStigler]

Thanks for this great package!

Would it be possible that summary(., relative=TRUE) prints relative time with respect to the minimum of each parameter group when using press? If one uses different input size as below, it is clear that with larger sized, timing will be longer than smaller sizes, yet one is more interested in computing within size relative timings?

Thanks!

create_df <- function(rows, cols) {
  as.data.frame(setNames(
    replicate(cols, runif(rows, 1, 1000), simplify = FALSE),
    rep_len(c("x", letters), cols)))
}

out <- bench::press(
  rows = c(1000, 10000),
  cols = c(10),
  {
    dat <- create_df(rows, cols)
    bench::mark(
      min_time = .05,
      bracket = dat[dat$x > 500, ],
      which = dat[which(dat$x > 500), ],
      subset = subset(dat, x > 500)
    )
  }
)
#> Running with:
#>    rows  cols
#> 1  1000    10
#> 2 10000    10

summary(out, relative=TRUE)
#> # A tibble: 6 x 8
#>   expression  rows  cols   min median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr> <dbl> <dbl> <dbl>  <dbl>     <dbl>     <dbl>    <dbl>
#> 1 bracket     1000    10  1.10   1.10      6.49      2.09      Inf
#> 2 which       1000    10  1      1         6.97      1         NaN
#> 3 subset      1000    10  1.35   1.41      5.19      2.59      Inf
#> 4 bracket    10000    10  7.22   7.39      1        21.5       Inf
#> 5 which      10000    10  3.70   2.96      2.12     10.6       Inf
#> 6 subset     10000    10  8.28   6.28      1.13     23.6       Inf

^{Created on 2021-02-09 by the reprex package (v1.0.0)}

[DavisVaughan]

For future us: This is complicated by the fact that bench::press() returns an object with the same class as bench::mark(), so there is no way for summary.bench_mark() to understand that the relative results need to be computed within each parameter combination.

• bench::press() could return a bench_press subclass
• bench::press() could add on a parameters attribute that summary.bench_mark() knows to look for (but knowing when to invalidate this could be hard, like if the tibble is sliced by [).