S3 Invalid Range 416 error when trying to read empty directory

[jeet23]

The readFileMultipart API is unable to read files from an empty directory and errors out with an S3 exception.

Steps to reproduce: Let's say we have a bucket localbucket has folder called directory but there is no files inside it.

val result = 
  s3.readFileMultipart(
          bucket = BucketName("localbucket"),  
          key = FileKey("localbucket/directory/"),
         partSize = PartSizeMB(25)
  )

Output:

Actual:

software.amazon.awssdk.services.s3.model.S3Exception: 
The requested range is not satisfiable (Service: S3, Status Code: 416, Request ID: xyz, 
Extended Request ID: ...

Expected: Empty Stream with new Array[Byte](0)

[semenodm]

Since we don't know if the key is not present in advance, the way to fix this is to check the exception and return an empty stream if the exception indicates a non-existing key. Looking at this exception, it needs to be made clear how to identify that this key does not exist. I want to avoid parsing the exception message to figure it out.

[jeet23]

According to AWS docs, a 416 should be accompanied with InvalidRange error code.

We could catch it like:

case ex: S3Exception =>
   if(ex.awsErrorDetails().errorCode() == "InvalidRange")
      # return empty stream

[semenodm]

I am not convinced that InvalidRange ALWAYS means no key to read.

[jeet23]

I don't see any other status codes/ error messages in S3 docs that return InvalidRange. Do you have any other ideas @semenodm ?

[jeet23]

@semenodm do you have any updates here?