aws-ses-actions: S3 ReceiptRule cannot be created

[Zetten]

Describe the bug

When calling IReceiptRuleSet. addRule with an S3 action it doesn't seem possible to deploy the rule.

Expected Behavior

The receipt rule with S3 action is added to the requested ruleset.

Current Behavior

Deployment fails with a Could not write to bucket error:

1:36:13 PM | CREATE_FAILED | AWS::SES::ReceiptRule | TestRuleSetStoreToBucketRule3E41D5CF Could not write to bucket: reprosess3rulestack-testemailstoref58b593c-dxh45g1m3y6b (Service: AmazonSimpleEmailService; Status Code: 400; Error Code: InvalidS3Configuration; Request ID: 817f5520-748b-4bae-b347-ec68df52b675; Proxy: null)

Reproduction Steps

I've created a reproducing project here: https://github.com/Zetten/repro-ses-s3-rule

The S3 bucket is set up with my project's defaults, but the error is identical without any other props, i.e. it fails even if I set no encryption, versioning, lifecycle rules.

The relevant call to addRule:

    const receiptRuleSet = ses.ReceiptRuleSet.fromReceiptRuleSetName(this, 'TestRuleSet', 'TestRuleSet');

    receiptRuleSet.addRule('StoreToBucketRule', {
      receiptRuleName: 'StoreToBucketRule',
      recipients: [props.recipient],
      actions: [
        new actions.S3({
          bucket: emailStoreBucket,
          objectKeyPrefix: 'emails/',
        }),
      ],
      enabled: true,
    });

Possible Solution

The same error is received in the AWS SES console when adding a receipt rule without having previously set up the access policy.

Therefore the problem may be related to resource ordering - the S3 BucketPolicy is perhaps not created before the ReceiptRule. I note that this sounds very similar to https://github.com/aws/aws-cdk/issues/3726 which was resolved some years ago. The S3 action seems to carry the required policy itself, so perhaps it's a missing dependency?

Additional Information/Context

The issue does not seem explicitly related to the use of an existing ReceiptRuleSet (i.e. ReceiptRuleSet.fromReceiptRuleSetName) - the same error is observed when creatng a new one with const receiptRuleSet = new ses.ReceiptRuleSet(this, 'StoreToBucketRuleSet');.

Additionally, manually setting a dependency as mentioned in #3726 via:

const cfnBucketPolicy = emailStoreBucket.node.findChild("Policy").node.findChild("Resource") as s3.CfnBucketPolicy;
receiptRuleSet.node.addDependency(cfnBucketPolicy);

fails due to a circular dependency (even with autoDeleteObjects: false):

Circular dependency between resources: [TestEmailStorePolicyF234249E, TestRuleSetStoreToBucketRule3E41D5CF, TestEmailStoreAutoDeleteObjectsCustomResource06AE7680]

or

Circular dependency between resources: [TestEmailStorePolicyF234249E, TestRuleSetStoreToBucketRule3E41D5CF]

CDK CLI Version

2.141.0 (build 3d1c06e)

Framework Version

No response

Node.js Version

18.20.2

OS

Linux

Language

TypeScript

Language Version

TypeScript (5.4.5)

Other information

No response

[khushail]

Hi @Zetten , thanks for reaching out. I am able to repro this error while deploying with CDK 2.141.

But I noticed this PR caused changes in S3 policy in CDK 2.139 which might have led to the error being seen here. So i tried deploying with CDK V2.138.0 and the deployment succeeded. Sharing the code -

    const bucket = new s3.Bucket(this, 'Bucket10061');
   
    const ruleSet = new ses.ReceiptRuleSet(this, 'RuleSet', {
      dropSpam: true,
    });

    const awsRule = ruleSet.addRule('Aws', {
      recipients: ['aws.com'],
    });

    ruleSet.addRule('StoreToBucketRule', {
      receiptRuleName: 'StoreToBucketRule',
      recipients: ['aws.com'],
      actions: [
        new actions.S3({
          bucket: bucket,
          objectKeyPrefix: 'emails/',
        }),
      ],
      enabled: true,
    });

let me know if deploying with CDK 2.138 also works for you

[CZhang1997]

Hi @khushail , 2.138 does not work for me, but I found another workaround by create the bucket in somewhere else, then use

    this.incomingEmailEventBucket = Bucket.fromBucketArn(
      this,
      "incoming-email-event-bucket-arn",
      s3BucketsStack.incomingEmailEventBucket.bucketArn
    );

to apply to the S3 action

[sudoplatform-engineering]

I'm seeing the same issue, rolling back to 2.138.0 addresses the issue for me. I'm creating the bucket in the same stack as my receipt rules.

I wonder if a way to inhibit automatic policy being created would help here? Or deferring whatever test that checks that S3 is able to be written to is deferred until after the bucket policy is set. Though I think this is an SES API so that might not be possible.

[Zetten]

Hi @khushail, thanks for the suggestion. For me downgrading to [email protected] and [email protected] didn't solve the issue.

Our project adopted a slightly different workaround from @CZhang1997, although still using a separate stack - we create the ruleset and bucket, and manually create a bucket policy which allows a wildcard of receipt-rule-set/MyReceiptRuleSet/receipt-rule/*. Then the app stack can successfully provision its own rules. It's a bit clunky but it follows patterns we're using elsewhere for resource reuse.

I agree with @sudoplatform-engineering's suggestion that being able to disable the policy creation should allow other workarounds with manual dependency ordering (as well as custom policies in general), but (perhaps naively) it feels like it should be possible with the implicit creation.

[jonathanbodart]

At our side, just as for @sudoplatform-engineering , downgrading to [email protected] and having the ingestion bucket in the same stack has the Receipt Rule resolved the issue.

[guckin]

We see the same bug on our side. Downgrading to [email protected] seems to resolve the issue.

[AbbadV]

Seeing the same issue. I downgraded from aws-cdk-lib==2.143.0 to aws-cdk-lib==2.138.0 which seemed to fix.

[amine-mf]

Downgrading is "workaround", not a fix. It locks you on a specific version. Looks important enough to be handled quickly IMHO.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[shikha372]

Update

We've merged the revert PR ( Reference here ) to fix this issue, this will be released as part of version 2.143.1. We'll be doing the patch release soon.

[shikha372]

Closing Notes

Fix released in version https://github.com/aws/aws-cdk/releases/tag/v2.143.1

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.

[aws-cdk-automation]

Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.