Developer Guide: Document Support for Amazon S3 Object Integrity Features

[skmcgrail]

A little bit more nuanced, but I think this highlights that it is worth creating a developer guide documentation page to help clarify the behaviors when using the S3 client and checksums.

• If a user specifies an explicit checksum value in either the ChecksumCRC32, ChecksumCRC32C, ChecksumSHA1, or ChecksumSHA256 fields then the SDK will send that value specified using a standard HTTP request header using the appropriate x-amz-checksum-* header for each. This is not affected by whether the HTTP or HTTPS scheme is used. The request will be signed with SigV4 or SigV4a regardless of the scheme, the only distinction is whether the payload body hash is computed or not for the x-amz-content-sha256 header. For HTTP schemes this will be the computed sha256 hash for the PutObject body, otherwise for HTTPS this will be UNSIGNED-PAYLOAD. If the payload body hash needs to be computed for sending to a non-HTTPS endpoint, then the body must be seekable so the SDK can compute the hash and reseek the body.
• If ChecksumAlgorithm field, and an algorithm Checksum* member is not set, then the SDK will compute the corresponding checksum for the request and will send it in the appropriate x-amz-checksum-*. The only distinction is if the checksum is sent as a header or trailer based on the endpoint scheme
  • If the scheme is HTTPS the computed checksum will be sent in a trailer header block using aws-chunked encoding. It will also do so use an unsigned payload variant called STREAMING-UNSIGNED-PAYLOAD-TRAILER which does not require signing each aws-chunk. The Go SDK only supports this variant of aws-chunked, which is why computed checksums sent via a aws-chunked trailer requires an HTTPS endpoint in this SDK. For trailer streaming the content-length of the PutObject body must be known. So either the PutObject request specifies ContentLength field, the body implements interface{ Len() int }, or the body is seekable so SDK can figure out the length to be sent. The SDK may add support for the signed aws-chunked variants (e.g. STREAMING-AWS4-HMAC-SHA256-PAYLOAD-TRAILER) in the future as to support sending of signed checksum trailers to non-HTTPS endpoints. That feature is supported in SDKs that had already supported signed aws-chunked payloads previously.
  • If the scheme is HTTP then the SDK will fallback to to a behavior similar to the provided checksum case, but instead the SDK will compute the value (thus the body must be seekable), and attach it as a header similar to the provided checksum case.

Originally posted by @skmcgrail in https://github.com/aws/aws-sdk-go-v2/issues/1667#issuecomment-1113754596

[petitout]

@skmcgrail, with the golang sdk, It is currently not possible to use Amazon S3 Object Integrity Features with presigned URL because of this open bug : https://github.com/aws/aws-sdk-go-v2/issues/1566

[github-actions[bot]]

This issue is now closed. Comments on closed issues are hard for our team to see. If you need more assistance, please open a new issue that references this one.