redshiftTools icon indicating copy to clipboard operation
redshiftTools copied to clipboard

Published 20 hours ago • verified publisher icon sicarul

Download Function

Open niklasvm opened this issue 5 years ago • 3 comments

This is a feature request more than an issue.

I'm interested in a function that can be used to download the contents of a table or results of a query using the reverse of the pattern used for rs_append_table. i.e. Redshift --> S3 --> Local

My main motivation is that a normal download via a SELECT query can take very long for large results.

niklasvm avatar Jun 15 '19 13:06 niklasvm

Have you been spying on me? 😝 I swear to you i had the same idea a few weeks ago, i will get to it as soon as i can.

sicarul avatar Jun 16 '19 00:06 sicarul

Awesome thank you!

niklasvm avatar Jun 16 '19 05:06 niklasvm

Might be useful for the project, I've created a unloadToS3() function that'll do the first step of this feature. See my fork if interested. The feature I needed was the ability to move tables from one RS cluster to another RS cluster (via S3, no local). Essential for huge tables. My rs_move_table function is still very hacky at the moment but it works for me.. All and all thanks for the package @sicarul ! Very useful

BenoitFayolle avatar Oct 04 '19 16:10 BenoitFayolle