Retrieving data from an S3 bucket

[zachasman]

I'm currently using the aws.s3 package in R to extract data from various CSV's in one of my AWS S3 buckets. The data is stored as seen below.

What I'm trying to do is extract ALL of the data currently in .csv format. So if I were just doing the first file there it would look as such:

data <- aws.s3::s3read_using(read.csv, object = "s3://test-testpilot/09242021-testpilot-michigan-script_results-09-24-2021-871107619.csv")

But the problem is, I want to extract all of the files and put them into three separate data frames. The first dataframe would be all the files that contain -script_results-. The second dataframe would be any file with just -results- in it, and the final dataframe would be any .csv file with -callers-.

What would be the most effective way to go about doing this?

$Contents
    Key:            09242021-testpilot-michigan-script_results-09-24-2021-871107619.csv 
    LastModified:   2021-09-25T01:36:35.000Z 
    ETag:           "7ee0a56df05121e815b2f1bc2fba819d" 
    Size (B):       28738 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-nevada-callers-09-24-2021-937053503.csv 
    LastModified:   2021-09-25T07:36:56.000Z 
    ETag:           "3363a2788d9f3c9822a0a634c7c07095" 
    Size (B):       889 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-nevada-results-09-24-2021-704158614.csv 
    LastModified:   2021-09-25T07:36:57.000Z 
    ETag:           "c69d06426351db42d03a4c0616e526eb" 
    Size (B):       85399 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-nevada-script_results-09-24-2021-780474261.csv 
    LastModified:   2021-09-25T07:36:57.000Z 
    ETag:           "b6be083391b484a5ab2a2574a1c6fa4a" 
    Size (B):       19051 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-new-hampshire-callers-09-24-2021-779182855.csv 
    LastModified:   2021-09-25T01:55:30.000Z 
    ETag:           "867c55d68310e9a55d95f1b1b5ef5b96" 
    Size (B):       1903 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-new-hampshire-results-09-24-2021-574695815.csv 
    LastModified:   2021-09-25T01:55:31.000Z 
    ETag:           "da4026197af765d0ef076f0f61387903" 
    Size (B):       96827 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-new-hampshire-script_results-09-24-2021-830415712.csv 
    LastModified:   2021-09-25T01:55:31.000Z 
    ETag:           "0b8942e4bb80787cb6e4bbda10dfcc3c" 
    Size (B):       19509 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-north-carolina-callers-09-24-2021-728851613.csv 
    LastModified:   2021-09-25T01:42:34.000Z 
    ETag:           "b72c362480a1ae171f6edfc3cca861b3" 
    Size (B):       2316 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-north-carolina-results-09-24-2021-690797258.csv 
    LastModified:   2021-09-25T01:42:35.000Z 
    ETag:           "0ab9a52dff07518668fe5a36e5ca16c7" 
    Size (B):       636831 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-north-carolina-script_results-09-24-2021-552436920.csv 
    LastModified:   2021-09-25T01:42:35.000Z 
    ETag:           "79a4492084a57ee07fbb0fc4550e6ec9" 
    Size (B):       88138 
    Owner:          
    Storage class:  STANDARD 
    
    $Contents
    Key:            09242021-testpilot-pennsylvania-callers-09-24-2021-695602875.csv 
    LastModified:   2021-09-25T01:41:51.000Z 
    ETag:           "bc9e2f80431ba40352f86a759b6a697f" 
    Size (B):       1750 
    Owner:          
    Storage class:  STANDARD