Check both MD5 locations for S3 KMS support.

[Melraidin]

Fixes https://github.com/aptly-dev/aptly/issues/1117

Requirements

All new code should be covered with tests, documentation should be updated. CI should pass.

Description of the Change

If the S3 bucket used to house a repo has KMS encryption enabled then the etag of an object may not match the MD5 of the file. This may cause an incorrect error to be reported stating the file already exists and is different.

A mechanism exists to work around this issue by using the MD5 stored in object metadata. This check doesn't always cover the case where KMS is enabled as the fallback is only used if the etag is not 32 characters long.

This commit changes the fallback mechanism so that it is used in any case where the object's etag is not 32 characters or if the S3 bucket has encryption enabled for new objects by default.

Checklist

• [ ] unit-test added (if change is algorithm)
• [ ] functional test added/updated (if change is functional)
• [x] man page updated (if applicable)
• [x] bash completion updated (if applicable)
• [x] documentation updated
• [x] author name in AUTHORS

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 57.89474% with 8 lines in your changes missing coverage. Please review.

Project coverage is 74.62%. Comparing base (f0bf519) to head (8e3c12f).

Files	Patch %	Lines
s3/public.go	57.89%	6 Missing and 2 partials :warning:

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #1167      +/-   ##
==========================================
- Coverage   74.63%   74.62%   -0.01%     
==========================================
  Files         144      144              
  Lines       16299    16312      +13     
==========================================
+ Hits        12164    12173       +9     
- Misses       3188     3192       +4     
  Partials      947      947              

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[neolynx]

I am getting a build error after rebase on master:

s3/public.go:109:48: not enough arguments in call to storage.s3.GetBucketEncryption
have (*"github.com/aws/aws-sdk-go-v2/service/s3".GetBucketEncryptionInput)
want ("context".Context, *"github.com/aws/aws-sdk-go-v2/service/s3".GetBucketEncryptionInput, ...func(*"github.com/aws/aws-sdk-go-v2/service/s3".Options))
s3/public.go:115:4: invalid operation: cannot indirect output.ServerSideEncryptionConfiguration.Rules[0].ApplyServerSideEncryptionByDefault.SSEAlgorithm (variable of type "github.com/aws/aws-sdk-go-v2/service/s3/types".ServerSideEncryption)

could you try to fix the build on master ?

[neolynx]

Rebased on master, some fixes and updated test: https://github.com/aptly-dev/aptly/pull/1272

Could you maybe check if this works for you ? Also, why are two s3 calls to /test?encryption= needed (https://github.com/aptly-dev/aptly/pull/1272/files#diff-7ed984d7519264eda1b9ba70e1540c82b75bb2f53ff87436a3520fc5ee2d9436R350) ?

[neolynx]

abandonned https://github.com/aptly-dev/aptly/pull/1272, continuing here