rdatacite icon indicating copy to clipboard operation
rdatacite copied to clipboard

Cache DataCite responses locally

Open katrinleinweber opened this issue 6 years ago • 5 comments

It would useful for teams of bibliometricians to be able to share/sync a cache of their DataCite queries. Analogous to pybliometrics Python package for example.

Which of the "cache"-related CRAN packages seems most usable to ensure that? I tried it with https://github.com/r-lib/memoise/issues/106 but that approach seems to have failed.

katrinleinweber avatar Mar 06 '20 11:03 katrinleinweber

Thanks for the issue @katrinleinweber - What exactly do you want to cache? The HTTP response with headers and the raw response body? Or the parsed response as a data.frame/list? Or some other format?

sckott avatar Mar 06 '20 22:03 sckott

I think pybliometrics caches the entire HTTP response.

I presume the risk of discarding potentially useful metadata (maybe an etag for requesting a refresh later?) is not worth the small saving in storage space. Maybe @pybliometrics-dev can comment about their thinking what to cache? I tried to find an explanation in the blame trail of the above-linked lines and read PR 17.

katrinleinweber avatar Mar 07 '20 12:03 katrinleinweber

thanks @katrinleinweber

I have been tinkering with this pkg in development https://github.com/ropenscilabs/webmiddens exactly for the use case of caching http requests/responses with expiry, etc. I'll try to get that working

sckott avatar Mar 10 '20 19:03 sckott

@katrinleinweber install the version on middens branch remotes::install_github("ropensci/rdatacite@middens")

The README https://github.com/ropensci/rdatacite/tree/middens#caching has some instructions on use.

The cached data is persistent on disk - in binary format to save disk space - it's not human readable really - its a feature I want to add to webmiddens though. You can set the cache path folder - see ?dc_caching

sckott avatar Mar 10 '20 23:03 sckott

Awesome, thank you! I hope I'll have time to test it this March, but there is a risk that I won't.

katrinleinweber avatar Mar 11 '20 12:03 katrinleinweber