unconf17
unconf17 copied to clipboard
ropensci
AWS packages
Now that all AWS Java SDK jar files are on CRAN, we could relatively easily build wrapper R packages to interact with different Amazon Web Services.
Example implementation: AWS Key Management Service
It would be nice to discuss eg with @leeper if using the Java SDK provides faster development and hopefully less maintenance than interacting with the REST API, especially if we are not only building on the SDK, but using specific Java clients as well, like in the R Kinesis client.
Would love to discuss. I think there are trade-offs here - basically RJava is perpetual pain versus having to basically build a native R SDK - and I can imagine there's room for both, especially if they are designed to work well (e.g., as drop-in replacements for one another).
Just chipping in - would love to collaborate on this, although I'm with Thomas on the tradeoffs. Generally my preferences would go:
- Whizzy SDK
- RESTful SDK
- Anything that relies on RJava
So we might want to look at the C++ SDK, since Rcpp is much more common and less likely to shit the bed than RJava, or (yep) an R SDK (although I imagine that'd either be massive or an entire ecosystem of packages)
Thank you both! I agree rJava can be painful, but it has its advantages, eg
- being able to use the Java client apps built on the top of the SDK
- ease of installation on all platforms, dependency handling
- very simple API when it comes to passing simple things to Java, like a string or a number
Anyway, looking forward to collaborate on this! Eg a reimplementation of the AWR.KMS package using the C++ SDK would be I think a very useful resource as a minimal example on how to use Rcpp VS rJava, although probably there are some other tutorials already out there.
Reiterating what has already been said, a C++ solution is definitely preferred to a Java one in a vacuum.
I could be misreading the documentation for the C++ SDK, but it seems it needs to be compiled with either Visual Studio (not kosher with Rcpp), or cmake. It appears that with some work you can compile code in an R package using cmake: https://github.com/forexample/r-cmake
But I doubt the cmake route would ever fly on CRAN. Regardless I think it is worth exploring.
Thanks for the updates, @carlganz, and I'm looking forward to any help/collaboration on this. Although the rJava version is already on CRAN and we have to stick with the jars for some purposes (eg the Kinesis MultiLangDaemon), but I'd be very happy to see a viable C++ approach. We will also have to keep in mind that there's a 5Mb cap on CRAN packages, for which there's a workaround in place for the rJava approach (downloading jars at installation time), we will have to figure out how to do that in the Rcpp way, although I'm pretty sure there are well established strategies to resolve that problem.
Weeeelll sorrta, the well-established strategies uniformly make my internal security pedant wail and gnash teeth, but they do exist, yeah.
I'm still very interested in this project, so not closing the ticket, but I'm having a bit more love for #13 right now
How would/can you test different AWS services, e.g. EC2 - because I might imagine you need some money to spend ...(update & note to self:amazon free trial!!!)
