redshiftTools
redshiftTools copied to clipboard
sicarul
Archived on CRAN: Use paws R Library as dependency
In order to put it back to CRAN, I would use Python's boto3 with reticulate or R's paws Library:
https://cran.r-project.org/web/packages/paws/index.html
Hey @JakeRuss i'm no longer using Amazon Redshift which complicates mantaining the package for me. If someone wants to take over i'm happy to help with the transition
@sicarul I would have interest as I use redshift daily for work. Discussing with my employer whether we'd take over the project. I will reach out to you via email.
I have used boto3 via reticulate for copying data between local and S3. If you are going to python then I propose using one of the python solutions for uploading and downloading to/from redshift
I plan to take over the maintainer-ship of this package soon, unless someone beats me to it.
My plan is to use AWS' R SDK {paws} to keep the package simpler to implement and maintain. But if someone else takes over before I do and strongly prefers going the {reticulate}/{boto3} route, by all means do so.
For anyone wishing to follow my progress, I have started refactoring this package over at https://github.com/RedOakStrategic/redshiftTools/tree/redoakjake. I will have an initial working version soon. @russellpierce
Note that aws.s3 package is available again in CRAN, and maintanied by Simon Urbanek (R Core), so redshiftTools it could be re-uploaded to CRAN.
Best, Juan
El lun., 13 abr. 2020 22:14, Jake Russ [email protected] escribió:
For anyone wishing to follow my progress, I have started refactoring this package over at https://github.com/RedOakStrategic/redshiftTools/tree/redoakjake. I will have an initial working version soon. @russellpierce https://github.com/russellpierce
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/sicarul/redshiftTools/issues/53#issuecomment-613075920, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHA4WJ4XLPEAD2MSFHOGYMLRMNXDTANCNFSM4KPUV3PQ .
I appreciate your note, @jtelleriar. I did see that @su rescued the {aws.s3} package as a temporary measure to get the package back on CRAN, but long term, {paws} is probably a better choice.
It's @s-u XD
El mar., 14 abr. 2020 4:15, Su [email protected] escribió:
Don't know what Su you guys are talking about but it's not me.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sicarul/redshiftTools/issues/53#issuecomment-613188192, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHA4WJ3ZYZW4JAT4C7SZADLRMPBLTANCNFSM4KPUV3PQ .
Yes, that was me ;). It depends on your use-cases, if you don't mind having Python dependency with the associated issues (can only embed one runtime, the usual Python version nightmare and foreign syntax) then paws has the benefit being able to leverage Amazon's work directly. With aws.* you get a native solution without extra dependencies, but the functionality is limited to what we implement in R. The community behind aws.* is picking up now but, obviously, we can't match Amazon ;).
There is no dependency on python with {paws}. While I applaud you @s-u and the others in the R community for stepping up to rescue {aws.s3}, I am going to press forward with {paws} for the new internals of {redshiftTools}.
@JakeRuss oh, sorry, I was reading that it uses reticulate to wrap boto3 - my bad. The API looks like an automated translation. There is pretty much zero information about the project so it's hard to know whom to trust ;). I looked now and I see it is also a wapper of curl. Thanks.