curl
curl copied to clipboard
Safe requests with session cookies
Let us assume I have to post to a page requiring a login. The code could be similar to:
## Login
h <- new_handle()
handle_setopt(h, postfields = "user=name&pass=mypw")
res <- curl_fetch_memory(login_url, h)
## Post
handle_reset(h)
options <- list(post = TRUE, postfields = "a=1&b=2")
handle_setopt(h, .list = options)
res <- curl_fetch_memory(post_url, h)
However, the manual says:
The safest way to perform multiple independent requests is by using a separate handle for each request. There is very little performance overhead in creating handles.
How to preserve session cookies, then? One way could be to obtain the cookies from handle_cookies
and adding them to the new handle via setheaders
.
cc <- handle_cookies(h)[c("name", "value")]
Cookie <- paste(cc$name, cc$value, sep = "=", collapse = "; ")
agent <- curl_fetch_memory('https://httpbin.org/user-agent')
agent <- strsplit(rawToChar(agent$content), '"')[[1]][4]
hds <- list(`User-Agent` = agent, Cookie = Cookie)
h <- new_handle()
handle_setheaders(h, .list = hds)
handle_setopt(h, .list = options)
res <- curl_fetch_memory(post_url, handle = h)
Is this type of approach correct? I haven't found any practical example in the docs.
Also, cookies added manually, with handle_setheaders
or handle_setopt
, do not appear in handle_cookies
output.
Retaining cookies is the one case where it makes sense to re-use a handle. See https://cran.r-project.org/web/packages/curl/vignettes/intro.html#Reading_cookies
Thank you, one last thing, please.
Say I would like to replicate a browser request, for testing purposes. To accomplish this, I need to copy the session cookies from the Web Developer Tools. How can I attach them to the handle?
I have tried with handle_setopt(handle, cookie = ...)
, but handle_cookies(handle)
is unaffected.