Large difference in run-times of test files on being measured with testthatQuantity wrapper

[analyticalmonk]

@tdhock Hey Toby, this was what I was talking about.

The results

Run-time for entire file with original version

          test_name     seconds status branch           date_time
1  basic case works 0.001079901   pass master 2015-01-08 14:09:43
2 NULLs are dropped 0.001078013   pass master 2015-01-08 14:09:43
3       test-join.r 0.002519254   pass master 2015-01-08 14:09:43
4  basic case works 0.001080284   pass master 2015-01-08 14:03:37
5 NULLs are dropped 0.000931637   pass master 2015-01-08 14:03:37
6       test-join.r 0.002546496   pass master 2015-01-08 14:03:37

and Run-time for entire file with substituted code (testthatQuantity)

          test_name     seconds status branch           date_time
1  basic case works 0.001073032   pass master 2015-01-08 14:09:43
2 NULLs are dropped 0.000929690   pass master 2015-01-08 14:09:43
3       test-join.r 0.132559892   pass master 2015-01-08 14:09:43
4  basic case works 0.001076601   pass master 2015-01-08 14:03:37
5 NULLs are dropped 0.000931337   pass master 2015-01-08 14:03:37
6       test-join.r 0.132972392   pass master 2015-01-08 14:03:37

The background

   seconds_file <- if(require(microbenchmark)){
      times <- microbenchmark(test = {
        source(temp_file1, local = T)
      }, times = 3)
      times$time/1e9
    } else {
      replicate(3, {
        time_vec <- system.time( {
          source(temp_file1, local = T)
        } )
        time_vec[["elapsed"]]
      })
    }

and

seconds_file2 <- {
      times <- microbenchmark(times = 3, source(file = temp_file2, local = T))
      times$time/1e9
    }

where the tempfiles 1 and 2 correspond to the non-substituted and substituted version of the test-code.

  t_lines <- readLines(test_path)
  q_lines <- sub("test_that(", "testthatQuantity(", t_lines, fixed = TRUE)
  temp_file1 <- tempfile()
  temp_file2 <- tempfile()
  writeLines(t_lines, temp_file1)
  writeLines(q_lines, temp_file2)

[tdhock]

any update?

[tdhock]

if you do this same test with the current version of Rperform what is the result? if it is OK now can you please explain what was the trick you used to get it to work?