Context-dependent fixture recording?

[sckott]

So in https://github.com/getwilds/rcromwell/issues/44 a use case has come up that it'd be nice to have context-dependent usage of cassettes.

Consider the following use case where jobs are submitted to some engine that makes widgets. However, it takes some time before the metadata on jobs is available after submission. This is not a great situation for tests.

vcr::use_cassette("job_metadata_prep", {
  res <- submit_job(some_input = tempfile())
})

# Needed only for recording new fixture
Sys.sleep(10)

vcr::use_cassette("job_metadata", {
  res <- job_metadata(res$id)
})

expect_s3_class(res, "tbl")

Right now when I want to re-record the cassette (or on the very first recording I did) I have to make sure the Sys.sleep is not commented out because the submit_job call kicks off some process that leads to some wait time before appropriate data will return from job_metadata. (of course you may want to test these different behaviors, but lets's assume we want to test the behavior of job_metadata when the metadata is present.

Consider the possibility of

add_sleep <- function(sleep = 10) Sys.sleep(sleep)

vcr::use_cassette("job_metadata_prep", {
    res <- submit_job(some_input = tempfile())
  }, 
  on_new_cassette = list(before = NULL, after = add_sleep),
  if_vcr_off = list(before = NULL, after = add_sleep)
)

vcr::use_cassette("job_metadata", {
  res <- job_metadata(res$id)
})

expect_s3_class(res, "tbl")

Where

• new params on_new_cassette and if_vcr_off both accept a list of two things: before and after (or other names?), where each accepts a function, formula list or NULL, just like httr2:::as_mock_function
• param on_new_cassette is ignored if it's not a new cassette - what should happen if re-recording? if it's a new cassette then the before callback is run for before recording, and the after callback after recording

I think this takes care of the 3 scenarios in the linked issue - making it so that tests could be run without any manual toggling on/off of sleeps

[sckott]

What are the definitions of before and after? Right now they are (based on work on my local machiune):

• before: after cassette insertion but before executing code
• after: after executing code

[hadley]

After #472, you could take an approach more like this:

local({
  local_cassette("job-submit")

  res <- submit_job(some_input = tempfile())
  
  repeat {
    res <- job_metadata(res$id)
    if (res$state == "finished") {
      break
    }
    Sys.sleep(1)
  }

})

The challenge would be skipping the sleep in the case when you're replaying the requests. I wonder if it would be worth exposing a cassette_state() function, so then you could do something like:

local({
  local_cassette("job-submit")

  res <- submit_job(some_input = tempfile())
  
  repeat {
    res <- job_metadata(res$id)
    if (res$state == "finished") {
      break
    }
    if (cassette_state != "replaying") {
      Sys.sleep(1)
    }
  }

})

[sckott]

I like the cassette_state() idea. Let's explore that ...