AWS S3: uploadMultipart* with S3Headers fails

[wi101]

Versions used

Alpakka version: 4.0.0 Akka version: 2.6.20

Expected Behavior

S3.multipartUpload and S3.multipartUploadWithHeaders with S3Headers should succeed as before in v3.0.4

Actual Behavior

When calling S3.multipartUpload or S3.multipartUploadWithHeaders with S3Headers there is an error.

akka.stream.alpakka.s3.S3Exception: The specified header is not valid in this context

When I tried to call S3.multipartUploadWithHeaders with an empty header it worked. An assumption: this change in completionSink might be the reason: https://github.com/akka/alpakka/pull/2844

[sikinmettugi]

Hi, I also faced the same issue after updating alpakka-s3 from 3.0.1 to 4.0.0. From my side the current dependency versions are: akka 2.6.19 alpakka 4.0.0 akka-http 10.2.9 (if this helps)

[hagerf]

Hi, just wanted to add that this is a breaking change also for us, we cannot use Alpakka 4.0.0 until this is fixed :(

[mvillafuertem]

Hi there, I also faced the same issue. Any workaround?

[cptwunderlich]

Alpakka version 5.0 is also affected.

[gipeshka]

The issue is apparently caused by sending metadata/custom headers with the multipart upload completion request.

[lvitaly]

Hi there, I also faced this issue. Has anyone had any luck fixing this issue with Alpakka 4.0.0?

[ramazanyich]

we also have the same issue. I checked the difference in code for multipartUpload and multipartUploadWithHeaders. Actually multipartUpload does construction of S3Headers. And one of them which is causing a problem is a call to withCannedAcl(cannedAcl). cannedAcl is defaulted to the CannedAcl.Private.

If I construct s3Headers without that cannedAcl and use multipartUploadWithHeaders then upload is successful. here is the code which worked:

    val s3Headers = S3Headers.empty
      .withMetaHeaders(MetaHeaders(Map()))
      .withOptionalServerSideEncryption(None)
     
 S3.multipartUploadWithHeaders(
      bucket,
      fileKey,
      ContentType.parse(contentType).getOrElse(ContentTypes.`application/octet-stream`),
      s3Headers = s3Headers
    )

if you do not need server side encryption and metadata then you can just do call multipartUploadWithHeaders without s3Headers (default is s3Headers.empty in that case).

Question is why that default cannedAcl is causing a problem ? maybe it should be deleted from multipartUpload call ?

Probably it worked before 4.0.0 because whole s3Headers were ignored and only serverSideEncryption of s3headers were used during multipart upload.

[sebastian-alfers]

We published 5.0.0+17-5228417f-SNAPSHOT in case anyone would like try out that snapshot.

[hagerf]

Thanks @sebastian-alfers. Would be great to test, but since the final version is going to be released in version >5.0.0 which is covered by BSL (I presume), I don't think we'll be able to use it in our company sadly. Or will some patch of 4.0.0 include this?

[sebastian-alfers]

I am sorry to hear that, but currently we do not plan to backport this fix.