R.cache
R.cache copied to clipboard
HenrikBengtsson
Add support for multiple caching layers
Do you foresee adding support for multiple caching backends? Would you accept pull requests if they added additional backends? I'm looking to contribute to a project that might grow towards cascading caching layers, e.g. ask local disk, if local disk doesn't have it, look to a Redis cache, if the Redis cache doesn't have it look to AWS S3, etc.
Thanks - this sounds like a useful extension, so yes, I'd certainly accept pull requests.
I'm not sure how clearly you envision the design at this stage, but, for my sake, would you mind sketching out the design and some use/test cases in order for me to see what you're aiming at? If to big for handling via issues, feel free to use the wiki instead.
How does this plan sound to you?
PS. I've had it on the to-do list to add support for also in-memory caching/memoization. It sounds like this could fit the idea of multi-layer caching as well.
PPS. While I remember, the master branch holds the latest version on CRAN, whereas the develop branch holds what's currently under development (functional and tested though). For completely new features, I typically work of feature/NNN branches, e.g. feature/cascading, which are eventually merged into develop and then into master. So, I guess what you're proposing should eventually go into an feature/NNN branch.
Thanks for the quick reply.
I hacked something together that partially accomplishes this task outside of the framework of R.cache (cascading between Redis and AWS S3). However, that initial draft was a manual process and it seems like what should probably be done is more akin to having object based backends that can be given a specified order.
Given that storr might already have some of the features I'm thinking about implementing, I think I'll give @richfitz a few weeks to see if his package comes together on CRAN (https://github.com/richfitz/storr/issues/11). Then I'll sit down, look through both, see where the easiest path to implementation is. If I come down on extending R.cache, then, your plan sounds good, and I'll write something up. Thanks for your time and consideration.
Upon further consideration, I think it makes more sense to handle this feature request with a light wrapper around R.cache in some other package that suggests R.cache than to try to embed it alongside R.cache. See https://github.com/richfitz/storr/issues/11 for additional thoughts.