helm-s3 icon indicating copy to clipboard operation
helm-s3 copied to clipboard

Does not seem to work with cross account s3 buckets

Open mmccord-mdbuyline opened this issue 5 years ago • 4 comments

I tried setting up the plugin to use cross account s3 buckets and it doesn't seem to work. Using the aws cli works just fine for fetching charts, but the plugin returns an access denied error.

mmccord-mdbuyline avatar Mar 11 '19 22:03 mmccord-mdbuyline

We are also seeing those problems. Once we have set a ACL to helm s3 push we get access denied when we try to publish new charts (upload chart to s3: upload object to s3: AccessDenied: Access Denied)

geNAZt avatar Mar 27 '19 17:03 geNAZt

Possibly related to #75 We noticed the same thing opening up cross-account access via bucket policy. I was looking at https://stackoverflow.com/questions/43722678/amazon-s3-file-permissions-access-denied-when-copied-from-another-account to troubleshoot and using the command:

aws s3 cp s3://helm-chart-bucket/ s3://helm-chart-bucket/ --recursive --acl bucket-owner-full-control

I was able to see a list of files that I did not have access to from the owner account. I purged those files and reindexed with helm s3 reindex repo_name. Note that this included an updated index.yaml and other, new chart revisions pushed by the secondary account.

We are in process of testing the solution, but I think this can be fixed by adding to the helm s3 push command, --acl bucket-owner-full-control flag to keep permissions tied to the bucket owner.

JRemitz avatar Jul 22 '19 15:07 JRemitz

So note/hint for anyone else running on this, in order to achieve this properly on a private bucket, I had to...

  • Create a bucket policy which allowed the other accounts to use this bucket
  • Ensure your bucket does not have encryption required/automatically and do not upload files with encryption (was too painful to get working multi-account)
  • Since the s3 push plugin does not support --grants I have to upload with --acl bucket-owner-full-control
  • Then I had to use a bit of a trick to copy in-place to add multi-account access via --grants per-file with the following command.
aws s3 cp --recursive s3://s3-reponame-here/helm-charts/ s3://s3-reponame-here/helm-charts/ --grants full=id=redacted_account_canonical_id_here full=id=redacted_account_2_canonical_id_here full=id=redacted_account_3_canonical_id_here --metadata '{"x-last-updated-at":"`date +%s`"}'

^ Note you may not want all your accounts to have full access as I do above, use "read" instead where relevant

AndrewFarley avatar Sep 15 '20 03:09 AndrewFarley

Thanks @AndrewFarley this worked great.

Sam-Lane avatar Jul 12 '21 14:07 Sam-Lane