drone-cache icon indicating copy to clipboard operation
drone-cache copied to clipboard

Unable to read from S3 buckets with TLS/SSL required

Open bbs-csel opened this issue 1 year ago • 1 comments

Describe the bug We've recently enforced TLS/SSL on all traffic to and from our S3 buckets including the one holding the cached artifacts from our Drone pipelines. After we've forced this, the plugin is unable to read and write from/to the bucket:

[IMPORTANT] restore cache, restore failed, 2 errors: download from
<[redacted]/7b28d59f43753746111f159b0d67bea8/.sbt> to <.sbt>, extract files from downloaded archive, pipe reader failed, get file from storage backend, pipe writer failed, get the object, AccessDenied: Access Denied\n\tstatus code: 403, request id: [redacted], host id: [redacted];
download from <[redacted]/7b28d59f43753746111f159b0d67bea8/.ivy2> to <.ivy2>, extract files from downloaded archive, pipe reader failed, get file from storage backend, pipe writer failed, get the object, AccessDenied: Access Denied\n\tstatus code: 403, request id: [redacted], host id: [redacted];

We've attempted, on the same node (EC2) with the same IAM role to read/write from the bucket using e.g. the AWS CLI and that works without issue. Same goes for e.g. Java applications using the AWS SDK for Java, so it seems related to this application and/or the Go SDK (but haven't been able to find any chatter around that SDK being broken).

The policy used on s3 is the one below, as described by AWS here: https://aws.amazon.com/premiumsupport/knowledge-center/s3-bucket-policy-for-config-rule/.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AllowSSLRequestsOnly",
            "Effect": "Deny",
            "Principal": "*",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::[bucket]/*",
                "arn:aws:s3:::[bucket]"
            ],
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                }
            }
        }
    ]
}

To Reproduce

  1. Create and S3 bucket with above policy
  2. Try to either restore or build a cache in the bucket
  3. See error in the plugin logs

Expected behavior The plugin reading and writing to the bucket unimpacted by the change.

bbs-csel avatar Aug 04 '22 10:08 bbs-csel

It seems like this might be related to the same issue described in https://github.com/meltwater/drone-cache/issues/144 which also seems to be around issues with lack of TLS by default.

bbs-csel avatar Aug 04 '22 10:08 bbs-csel