Document which permissions are required

[jariz]

We're currently suggesting people should give admin access to their IAM accounts in the README, this is probably not a very secure thing to suggest. We should figure out which exact permissions the plugin needs and document them, preferably through a JSON policy.

[YoshiWalsh]

I have an IAM policy with the exact permissions that I need, I'll get it later tonight. It might not cover all use cases though.

[jariz]

Should be a good starting point!

[YoshiWalsh]

Could I get some feedback on this? I'll make it pretty later, and maybe include a sample policy in JSON format too.

Required permissions:

Basic permissions

s3:HeadBucket s3:ListBucket (for the bucket) s3:GetBucketLocation (for the bucket) s3:GetObject (for all objects within the bucket)

If acl is configured as null (Recommended)

s3:PutObject (for all objects within the bucket)

If acl is not configured or is not null (Not recommended)

s3:PutObjectAcl (for all objects within the bucket)

If generateRoutingRules is true or not configured (Recommended)

s3:PutBucketWebsite (for the bucket)

If removeNonexistentObjects is true or not configured (Recommended)

s3:DeleteObject (for all objects within the bucket)

If you’re following the With CloudFront recipe and are doing automatic CloudFront invalidations (Optional)

cloudfront:CreateInvalidation cloudfront:GetInvalidation (optional)

If you are using gatsby-plugin-s3 to automatically create the bucket for you

s3:CreateBucket

[svvitale]

This worked for my first deploy. On subsequent deploys the upload succeeded, but S3 threw up a 403 when I tried to load the site in a browser. Updating all files to be public allowed the site to load correctly again. Removing acl: null and adding the s3:PutObjectAcl permission also resolved it.

[YoshiWalsh]

With acl: null, you need to add a bucket policy to allow public access. Please see this guide.

Configuring a bucket policy and disabling ACLs is recommended, because ACLs are considered legacy by Amazon. "As a general rule, AWS recommends using S3 bucket policies or IAM policies for access control. S3 ACLs is a legacy access control mechanism that predates IAM."

Source

[svvitale]

Yup, that worked for me. Thanks!

[jariz]

This looks good! @JoshuaWalsh Let's make a default sample policy and in the recipe guides, recommend people to add the additional perms to their policies.

[chris-erickson]

I have applied all these permissions on a new bucket, with acl: null and still am having issues getting past here:

- Retrieving bucket info...
✖ Failed.
  AccessDenied: Access Denied

I use a dedicated user for CI, and attached this policy to the user, and have the bucket policy for public access. I was able to deploy the first time, but not after that and I manually created the bucket. Do we have a concise set of steps and settings needed to get this working all the way on the first shot? I'm ok with giving this free reign on that particular bucket, the content is reproducible after all.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutAccountPublicAccessBlock",
                "s3:GetAccountPublicAccessBlock",
                "s3:ListAllMyBuckets",
                "s3:HeadBucket"
            ],
            "Resource": "*"
        },
        {
            "Effect": "Allow",
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::mysite.com/*",
                "arn:aws:s3:::staging.mysite.com/*",
                "arn:aws:s3:::mysite.com",
                "arn:aws:s3:::staging.mysite.com"
            ]
        }
    ]
}

As a suggestion, it might be nice to better print debug info for non-advanced users as I had to look through the source to find what action Retrieving bucket info... was trying to take, and then look up in the docs to find what that IAM policy might look like. Seems like we could print those details in the log output upon failure, perhaps?

I'm very thankful for this package, as it really helps solve one of the most boring, and boilerplate tasks for a S3 static site. I'm so close!

[jketcham]

@chris-erickson I just got this setup for me and was running into similar issues. This is the IAM policy I created:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "AccessToGetBucketLocation",
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation",
                "s3:CreateBucket" <- if you want this plugin to create the bucket if it doesn't exist
            ],
            "Resource": [
                "arn:aws:s3:::*"
            ]
        },
        {
            "Sid": "AccessToWebsiteBuckets",
            "Effect": "Allow",
            "Action": [
		"s3:PutBucketWebsite",
                "s3:PutObject",
                "s3:PutObjectAcl", <- if you don't want to set acl: null
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::your-bucket.com",
                "arn:aws:s3:::your-bucket.com/*",
                "arn:aws:s3:::staging.your-bucket.com",
                "arn:aws:s3:::staging.your-bucket.com/*"
            ]
        },
        {
            "Sid": "AccessToCloudfront", <- if you want to invalidate a cloudfront distribution cache
            "Effect": "Allow",
            "Action": [
                "cloudfront:GetInvalidation",
                "cloudfront:CreateInvalidation"
            ],
            "Resource": "*"
        }
    ]
}

[chris-erickson]

Thanks, somehow GetBucketLocation got lost, but I still am having the same error. For me I:

• Am using acl: null (because I might as well?)
• Not letting it create the bucket
• Not using CloudFront

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetBucketLocation"
            ],
            "Resource": "arn:aws:s3:::*"
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutBucketWebsite",
                "s3:PutObject",
                "s3:GetObject",
                "s3:ListBucket",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::mysite.com/*",
                "arn:aws:s3:::staging.mysite.com/*",
                "arn:aws:s3:::mysite.com",
                "arn:aws:s3:::staging.mysite.com"
            ]
        }
    ]
}

This policy is attached to the CI user which has access key/id added. 🤷‍♂️

[YoshiWalsh]

Hi @chris-erickson ,

My IAM policy also includes HeadBucket (not sure if this is necessary) and PutObjectAcl (which theoretically shouldn't be necessary with acl: null, but maybe this isn't working as expected). Could you please try adding these and seeing if it works correctly?

[jketcham]

@JoshuaWalsh the s3:ListBucket permission includes access for the HEAD Bucket operation in addition to the GET Bucket operation, you can see here: https://docs.aws.amazon.com/AmazonS3/latest/dev/using-with-s3-actions.html#using-with-s3-actions-related-to-buckets.

And in my IAM policy, I left out the PutObjectAcl permission since I'm setting acl: null and it's working fine.

@chris-erickson for my configuration, I was also able to set all the "Manage public access control lists (ACLs)" and "Manage public bucket policies" options to true and it's working for me.

If you're still having issues, what I did to see where I was failing was to install the aws-sdk npm package, set my AWS access/secret tokens as env vars (which the sdk sees) and open up a node shell, then try running each of the s3 commands in bin.ts to see which ones failed. For me, it was the s3.getBucketLocation command, because I didn't have that permission before. You could also just try using the aws-cli commands and make sure they work.

[jariz]

It should show the stacktrace on errors by default, but the AWS SDK is probably throwing some weird non default error object without a stacktrace.

[cesargdm]

This is perfectly working, but does not make the Cloudfront invalidation.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:ListBucket",
                "s3:DeleteObject",
                "s3:GetBucketLocation",
                "s3:PutBucketWebsite"
            ],
            "Resource": [
                "arn:aws:s3:::my-bucket.com",
                "arn:aws:s3:::my-bucket.com/*"
            ]
        }
    ]
}

[YoshiWalsh]

Something I just discovered: If you're allowing gatsby-plugin-s3 to create the bucket for you, you may also need PutBucketAcl. I imagine this isn't necessary if you've set acl: null in your config, but I haven't tested that yet.

[manterfield]

For frustrated developers arriving here from google...

One cause of the error:

- Retrieving bucket info...

✖ Failed.

  AccessDenied: Access Denied

Is a non-unique S3 bucket name. The plugin will attempt to get info on the bucket name you specified before doing its other work. If you don't own that bucket (since someone else already has the name you picked) it will quite rightly fail with a 403 and give you an error message that is about as much use as a chocolate tea pot.

[jgcmarins]

still facing:

✖ Failed.
  AccessDenied: Access Denied

after a gatsby-plugin-s3 deploy