helm-s3
helm-s3 copied to clipboard
Does not seem to work with cross account s3 buckets
I tried setting up the plugin to use cross account s3 buckets and it doesn't seem to work. Using the aws cli works just fine for fetching charts, but the plugin returns an access denied error.
We are also seeing those problems. Once we have set a ACL to helm s3 push we get access denied when we try to publish new charts (upload chart to s3: upload object to s3: AccessDenied: Access Denied)
Possibly related to #75 We noticed the same thing opening up cross-account access via bucket policy. I was looking at https://stackoverflow.com/questions/43722678/amazon-s3-file-permissions-access-denied-when-copied-from-another-account to troubleshoot and using the command:
aws s3 cp s3://helm-chart-bucket/ s3://helm-chart-bucket/ --recursive --acl bucket-owner-full-control
I was able to see a list of files that I did not have access to from the owner account. I purged those files and reindexed with helm s3 reindex repo_name
. Note that this included an updated index.yaml
and other, new chart revisions pushed by the secondary account.
We are in process of testing the solution, but I think this can be fixed by adding to the helm s3 push
command, --acl bucket-owner-full-control
flag to keep permissions tied to the bucket owner.
So note/hint for anyone else running on this, in order to achieve this properly on a private bucket, I had to...
- Create a bucket policy which allowed the other accounts to use this bucket
- Ensure your bucket does not have encryption required/automatically and do not upload files with encryption (was too painful to get working multi-account)
- Since the
s3 push
plugin does not support --grants I have to upload with--acl bucket-owner-full-control
- Then I had to use a bit of a trick to copy in-place to add multi-account access via --grants per-file with the following command.
aws s3 cp --recursive s3://s3-reponame-here/helm-charts/ s3://s3-reponame-here/helm-charts/ --grants full=id=redacted_account_canonical_id_here full=id=redacted_account_2_canonical_id_here full=id=redacted_account_3_canonical_id_here --metadata '{"x-last-updated-at":"`date +%s`"}'
^ Note you may not want all your accounts to have full access as I do above, use "read" instead where relevant
Thanks @AndrewFarley this worked great.