vcr icon indicating copy to clipboard operation
vcr copied to clipboard

Writing to disk does not yet work with httr2

Open sckott opened this issue 1 year ago • 6 comments

library(vcr)
library(httr2)

tmpdir <- tempdir()
invisible(vcr_configure(
  dir = file.path(tmpdir, "fixtures"),
  write_disk_path = file.path(tmpdir, "files")
))

my_file <- tempfile(tmpdir = file.path(tmpdir, "files"))

use_cassette(name = "stuff2", {
  request("https://google.com") |> req_perform(path = my_file)
})

readLines(my_file)
#> Warning in file(con, "r"): cannot open file
#> '/var/folders/qt/fzq1m_bj2yb_7b2jz57s9q7c0000gp/T//Rtmp8tCUFj/files/file6ec3313ed849':
#> No such file or directory
#> Error in file(con, "r"): cannot open the connection

Created on 2024-09-04 with reprex v2.1.1

sckott avatar Sep 04 '24 16:09 sckott

Furthermore, if you inspect the fixture yaml file, the response is in the body, not saved out to a file.

http_interactions:
- request:
    method: get
    uri: https://www.google.com/
    body:
      encoding: ''
      string: ''
    headers: []
  response:
    status:
      status_code: 200
      message: OK
    headers:
      accept-ch: Sec-CH-Prefers-Color-Scheme
      alt-svc: h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
      cache-control: private, max-age=0
      content-encoding: gzip
      content-length: '7179'
      content-security-policy-report-only: 'object-src ''none'';base-uri ''self'';script-src
        ''nonce-eL_DD1ONKovQUDjhElT_hw'' ''strict-dynamic'' ''report-sample'' ''unsafe-eval''
        ''unsafe-inline'' https: http:;report-uri https://csp.withgoogle.com/csp/gws/other-hp'
      content-type: text/html; charset=ISO-8859-1
      date: Tue, 18 Mar 2025 21:37:05 GMT
      expires: '-1'
      p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
      server: gws
      set-cookie:
      - AEC=AVcja2eeZQXcL9qyfRNfXB-FZR8dUNE6f-q9Lar1KhA4AyWdWEXXKkgZ-Fo; expires=Sun,
        14-Sep-2025 21:37:05 GMT; path=/; domain=.google.com; Secure; HttpOnly; SameSite=lax
      - NID=522=144u9gcJq_5KqVo8MRa9anSwWOU4K659bePu17YgKORFgDBS8mjnOkkTcxFEpsF9jgPBKZI6aji8mDmsvxE6I-nv2iCJU5YgjEETS1jkc3-h3QXDoHX-nCZJwj3zYioTVotWa5lUdtES3-CcvSZ9FSWZCrBUM2zWR_f_rvuu9NiiSh9ySDCOmXbWLIvB_iSYModTJjiVdN47QbgNorE;
        expires=Wed, 17-Sep-2025 21:37:05 GMT; path=/; domain=.google.com; HttpOnly
      x-frame-options: SAMEORIGIN
      x-xss-protection: '0'
    body:
      encoding: ''
      file: no
      string: |-
        <!doctype html><html itemscope="" itemtype="http://schema.org/WebPage" lang="en"><head><meta content="Search the world's information, including webpages, images, videos and more. Google has many special features to help you find exactly what you're looking for."

...

Aariq avatar Mar 18 '25 21:03 Aariq

Thanks @Aariq Yep, that's not the correct behavior. Thanks for the comment. Sorry I haven't got around to this yet.

sckott avatar Mar 19 '25 19:03 sckott

There's no way AFAICT to gather the existence or value of path with a http request using httr2::req_perform. I think it's because we're grabbing the request at this point https://github.com/r-lib/httr2/blob/main/R/req-perform.R#L81-L87 in the function, BEFORE path is handled and so there's no path information in the request object. crul and httr have path information in the request so those work.

@hadley know of any way to capture the path value (when used) from a httr2 request when mocking?

sckott avatar Apr 10 '25 22:04 sckott

Hmmm, why does it matter whether or not you save the body to a file? That's not a property of the request itself.

hadley avatar Apr 11 '25 15:04 hadley

Right, in ruby, python and other lang versions of vcr there's no such thing as writing to disk as part of the data that's written to and read from cassettes - it's always a separate task. But in R's http clients (httr, httr2, crul) we allow users to give a path to write the response to as part of the request incantation. So from a user's perspective it kinda is part of the request. Having said that, maybe you have a good point suggesting maybe removing support for writing to disk in cassettes

sckott avatar Apr 11 '25 15:04 sckott

I think there's a possible argument to be made that saving to a separate file would be a property of the cassette. I can certainly imagine cases where you'd want larger files to be saved separately, but I think that's really about the UI of vcr, not the underlying HTTP library.

hadley avatar Apr 11 '25 17:04 hadley