aws-cdk icon indicating copy to clipboard operation
aws-cdk copied to clipboard

lambda: EventSourceMapping in 2.159.0 introduced Tags, which are not supported in eu-west-2

Open shylaharild opened this issue 1 year ago • 3 comments

Describe the bug

Hello Team,

In our project, we updated the CDK version from 2.158.0 to 2.160.0 and encountered the error "Unsupported resource type for tagging or invalid ARN" when attempting to deploy the CDK stack.

Our project contains the following resources

  1. A Lambda Function
  2. An SQS Queue
  3. A Lambda SQS Event Source

The error occurs when the SQS Event Source is added to the Lambda function and inadvertently attempts to add tags to the EventSourceMapping resource, automatically created as part of this action. We do not set the tags, but it seems that the AWS CDK does that for us.

According to the CDK documentation, Event Source Mapping does not natively support tags. AWS documentation also confirms that tags are only supported for AWS::Lambda::EventSourceMapping resources as part of AWS CloudFormation stack-based groups.

This feature is available only in the us-east-1 region and has not yet been rolled out to other regions, such as eu-west-2. Since we use the ^ symbol in the package version numbers, the install command automatically upgrades to the latest version, which lacks the required tag support, thus causing the stack deployment to fail.

Regression Issue

  • [ ] Select this option if this issue appears to be a regression.

Last Known Working CDK Version

2.158.0

Expected Behavior

The CDK Stack deployment to successfully create all the resources mentioned above in the eu-west-2 region without error.

Fix used:

At the moment, we are using the workaround given in the Possible Solution section below or set the version to 2.158.0

Current Behavior

Stack deployment is successful when we hardcode the CDK version to 2.158.0 which means we will be missing out on other upgrades and features.

When the version is set to > 2.158.0, then the deployment fails in the eu-west-2 region.

Reproduction Steps

In your stack, add the following sample code to create the SQS Queue, Lambda Function and Event Source Mapping and deploy the stack to the eu-west-2 region. Make sure you set the CDK version to the latest (> 2.158.0)

const sqsQueue = new Queue(
  this,
  "events-queue",
  {
    queueName: "testing-queue",
    deliveryDelay: Duration.seconds(0),
    retentionPeriod: Duration.days(4),
    receiveMessageWaitTime: Duration.seconds(0),
    visibilityTimeout: Duration.minutes(16),
  },
);

const eventSource = new SqsEventSource(
  sqsQueue,
  {
    enabled: true,
  },
);

const lambda = new Function(this, 'Function', {
  runtime: Runtime.NODEJS_18_X,
  handler: 'index.handler',
  code: Code.fromInline('exports.handler = async function(event, context) {}'),
});

lambda.addEventSource(eventSource);

Possible Solution

This is not a fix but a workaround that we are using to solve this issue in the eu-west-2 region.

const sqsQueue = new Queue(
  this,
  "events-queue",
  {
    queueName: "testing-queue",
    deliveryDelay: Duration.seconds(0),
    retentionPeriod: Duration.days(4),
    receiveMessageWaitTime: Duration.seconds(0),
    visibilityTimeout: Duration.minutes(16),
  },
);

const eventSource = new SqsEventSource(
  sqsQueue,
  {
    enabled: true,
  },
);

const lambda = new Function(this, 'Function', {
  runtime: Runtime.NODEJS_18_X,
  handler: 'index.handler',
  code: Code.fromInline('exports.handler = async function(event, context) {}'),
});

const esm = new EventSourceMapping(this, "EventSourceMapping", {
  target: lambda,
  eventSourceArn: eventSource.queue.queueArn,
});

const cfnEsm = esm.node.defaultChild as CfnEventSourceMapping;
cfnEsm.addPropertyDeletionOverride("Tags");

Additional Information/Context

Related Github Issues:

#31532 cloudformation-coverage-roadmap #2137

CFN Resource Specification: us-east-1 region: CloudFormationResourceSpecification eu-west-2 region: CloudFormationResourceSpecification

CDK CLI Version

2.160.0

Framework Version

No response

Node.js Version

20.17

OS

Mac, Linux, Ubuntu

Language

TypeScript

Language Version

No response

Other information

No response

shylaharild avatar Oct 16 '24 08:10 shylaharild

The workaround may be effective for some cases where the stacks do not have dependencies. However, in our situation, we had other CloudFormation stacks dependent on this stack that includes the Event Source Mapping resource. As a result, applying the L1 construct workaround required us to destroy both this stack and all associated stacks before recreating them with the necessary changes.

While the workaround is technically functional, it involves destructive actions, making it impractical for higher environments, such as production, where stacks cannot easily be recreated.

sri-scc avatar Oct 16 '24 09:10 sri-scc

Hi @shylaharild , thanks for reaching out.

I see that tags are available in eu-west-2 as seen in this resource specification doc. Screenshot 2024-10-16 at 5 38 07 PM

Screenshot 2024-10-16 at 5 41 12 PM

I also tried to repro the issue by deploying the given code in eu-west-2 and it succeeds without any error.

PS. - I am using cdk version-2.162

Screenshot 2024-10-16 at 5 43 23 PM

Please feel free to correct me if something is missed. Also could you share more information how I can repro this in my account.

khushail avatar Oct 17 '24 00:10 khushail

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

github-actions[bot] avatar Oct 19 '24 04:10 github-actions[bot]

A note for others experiencing this: we discovered that if you don't have the necessary permissions for tagging a lambda, AWS returns a 400 error instead of a 403 -- it says "tags not supported" instead of "not authorized to tag".

Hi @shylaharild,

Thanks for opening this issue! After reviewing the details, we found some other discussions that may be duplicates or closely related:

Potential Duplicate Issues

  • #31532: Both issues report the same bug where AWS Lambda EventSourceMapping automatically includes Tags in CDK versions 2.159.0+, which fails in regions that don't support this property (eu-west-2 in the current issue, cn-north-1 in #31532). Both issues identify 2.158.0 as the last working version, show the same error pattern related to Tags not being permitted, and provide identical workarounds using addPropertyDeletionOverride("Tags"). The underlying technical cause is identical - CDK applying Tags to EventSourceMapping resources in regions where CloudFormation doesn't support this property.

  • #31965: This issue describes the same underlying problem where CDK automatically adds Tags to EventSourceMapping resources, causing deployment failures in regions that don't support tagging for these resources (eu-west-2 in the current issue, GovCloud in #31965). Both issues involve the same AWS::Lambda::EventSourceMapping resource with the same Tags property causing deployments to fail after CDK version 2.158.0. Both require similar workarounds to exclude or remove Tags from these resources.

Related Issues

  • #31987: This issue is related as it involves the same AWS::Lambda::EventSourceMapping resource and the same Tags property compatibility problem in different regions. While the current issue affects stack deployment in eu-west-2, #31987 affects the cdk migrate command in eu-south-2. Both stem from the same underlying cause where CDK is trying to use the Tags property on EventSourceMapping resources in regions where this property is not yet supported by CloudFormation.

  • #15947: This issue is related as it describes the broader pattern where CloudFormation spec additions (like Tags) can break deployments in regions with inconsistent support. The current issue with EventSourceMapping in eu-west-2 is a specific instance of this general pattern described in #15947. Both issues involve the CDK's automatic tagging mechanism causing deployment failures for resources in regions where Tags aren't fully supported, requiring similar workarounds to exclude specific resource types from tagging.

This message was generated automatically to help connect related conversations and improve discoverability.

If you feel this issue brings new or distinct information, feel free to add a comment to keep it open. Otherwise, we'll close this issue in 7 days if we don't recieve a response to help keep discussions consolidated.

Please react with 👍 or 👎 to let us know if this response was helpful!

Thank you for helping improve CDK! 🙌

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

github-actions[bot] avatar Aug 13 '25 20:08 github-actions[bot]