Improve error handling

[rtjohn]

trafficstars

When get_bucket works you can assign the results and get a list back. The list allows you to check various things about what was returned.

> x <- aws.s3::get_bucket("bucket_that_exists")
> x$Contents
Key:            impses.csv 
LastModified:   2017-05-25T20:34:45.000Z 
ETag:           "6a2ec42db68d72839f1900e1028f28cb" 
Size (B):       9804 
Owner:          itsupport 
Storage class:  STANDARD
> length(x)
[1] 10 # In this example

However, if the bucket name is invalid an error is thrown

> x <- aws.s3::get_bucket("bucket_that_does_not_exist")
List of 5
 $ Code      : chr "NoSuchBucket"
 $ Message   : chr "The specified bucket does not exist"
 $ BucketName: chr "bucket_that_does_not_exist"
 $ RequestId : chr "AF625877ED1C2914"
 $ HostId    : chr "Fm5TsAuP6B+6mrSM6WWDzczJVj22OxJnHs1X9Mrc+LTMq5AJRoerxwD3adJeGzu2d3NjI0Zs5IM="
 - attr(*, "headers")=List of 6
  ..$ x-amz-request-id : chr "AF625877ED1C2914"
  ..$ x-amz-id-2       : chr "Fm5TsAuP6B+6mrSM6WWDzczJVj22OxJnHs1X9Mrc+LTMq5AJRoerxwD3adJeGzu2d3NjI0Zs5IM="
  ..$ content-type     : chr "application/xml"
  ..$ transfer-encoding: chr "chunked"
  ..$ date             : chr "Mon, 31 Jul 2017 18:13:52 GMT"
  ..$ server           : chr "AmazonS3"
  ..- attr(*, "class")= chr [1:2] "insensitive" "list"
 - attr(*, "class")= chr "aws_error"
 - attr(*, "request_canonical")= chr "GET\n/bucket_that_does_not_exist/\nlocation=\nhost:s3-us-west-2.amazonaws.com\nx-amz-date:20170731T181352Z\n\nh"| __truncated__
 - attr(*, "request_string_to_sign")= chr "AWS4-HMAC-SHA256\n20170731T181352Z\n20170731/us-west-2/s3/aws4_request\n084acf445f38f37fae16666457f8bdd5e83e112"| __truncated__
 - attr(*, "request_signature")= chr "AWS4-HMAC-SHA256 Credential=AKIAJOFPY2ACOTABLUKQ/20170731/us-west-2/s3/aws4_request, SignedHeaders=host;x-amz-d"| __truncated__
NULL
Error in parse_aws_s3_response(r, Sig, verbose = verbose) : 
  Not Found (HTTP 404).

However, you can see that before the error a list is actually generated and this list contains useful diagnostic information such as "NoSuchBucket" or "bucket_that_does_not_exist". But because the error is thrown the list is never assigned. As my team is building functions that include get_bucket() it would be nice to surface that information so we can check more throughly that we are getting what we'd expect.

[leeper]

Thanks. This was actually the original behavior, but lots of people requested that it just error instead so I switched to this. I had it as an idea to include an option or something to toggle whether it was an error or returned an "error"-classed object, but haven't implemented it. Another idea is a logger, which might be easier to implement. I'll put some thought on it.

[cmwetherell]

Is there a way to suppress the error from being returned? I find the behavior interesting when running the code below:

aws.s3::head_object("file_that_doesnt_exist", bucket = "bucket_that_exists")[[1]]

Client error: (404) Not Found [1] FALSE

if (aws.s3::head_object("file_that_doesnt_exist", bucket = "bucket_that_exists")[[1]]) {print("Hi")}

Client error: (404) Not Found

EDIT: This is really funny. Please laugh at me. Note the "FALSE" output above, ha!

[mark-druffel]

Not sure if this is still being looked at, but I was dealing with a similar issue and was able to wrap the function in purrr::safely() and it provides meaningful error messages. Might avoid building it directly into the repo (reproducible example below).

safe_get_object <- purrr::safely(aws.s3::get_object)  
response <- safe_get_object(object = 'object', bucket = 'bucket', check_region = T, key = 'AWS_ACCESS_KEY_ID', secret = 'AWS_SECRET_ACCESS_KEY')