aws-cdk
aws-cdk copied to clipboard
aws
(aws-cdk-lib.aws_s3_notifications): (Configuration is ambigously defined even through prefixes are not overlapping)
Describe the bug
The S3 notification module is throwing an configuration is ambiguously defined error even though the prefixes do not overlap. This issue is resolved if I remove all currently defined notifications and re-add them along with the new one.
Example code
//2D Bounding Box Upload Notification
this.privateBucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(props.eventLambda), {
// Declared in s3_generic_map.py
prefix: "bounding_box_2d/",
});
//KV Rule Engine Notification
this.privateBucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(props.eventLambda), {
// Declared in s3_generic_map.py
prefix: "kv_rule_engine/input/",
});
Expected Behavior
I expect the relevant event notifications to be created on the relevant S3 bucket
Current Behavior
CloudFromation is throwing an error
notification configuration
Traceback (most recent call last):
File "/var/task/index.py", line 24, in handler
s3.put_bucket_notification_configuration(Bucket=props["BucketName"], NotificationConfiguration=config)
File "/var/lang/lib/python3.11/site-packages/botocore/client.py", line 553, in _api_call
return self._make_api_call(operation_name, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/botocore/client.py", line 1009, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type
Reproduction Steps
- Create a single event notification with some prefix on an S3 bucket
- Deploy changes
- Create another event notification on the same S3 bucket but with a different prefix
- Observe error thrown by CloudFormation
Possible Solution
No response
Additional Information/Context
Recently upgraded to "aws-cdk-lib": "^2.138.0" which might be causing this issue
CDK CLI Version
2.138.0
Framework Version
No response
Node.js Version
10.8.1
OS
Linux
Language
TypeScript
Language Version
No response
Other information
No response
@thenamanpatwari Good morning. Please share the following:
- If you are using Python or TypeScript to write CDK stack (I noticed code comment
// Declared in s3_generic_map.py). - Are you able to synthesize the CF template using
cdk synth?- What does it look like with respect to event notification perspective?
- What is the output of
cdk diff?
- Share the minimal end-to-end reproduction code for us to troubleshoot the issue.
Using the below CDK TypeScript code, I was able to deploy CF stack (first deploying one event notification and then another one as part of same stack):
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as s3n from 'aws-cdk-lib/aws-s3-notifications'
export class TypescriptStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const lambdaFunction = new lambda.Function(this, 'TestLambdaFunction',{
runtime: lambda.Runtime.NODEJS_18_X,
code: lambda.Code.fromAsset('lambda'),
handler: 'hello.handler'
});
const s3Bucket = new s3.Bucket(this, 'TestBucket');
s3Bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(lambdaFunction), { prefix: "bounding_box_2d/" });
s3Bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(lambdaFunction), { prefix: "kv_rule_engine/input/" });
}
}
const app = new cdk.App();
new TypescriptStack(app, 'TypescriptStack');
Thereafter, when I upload an object to S3, I see no error in the CloudWatch logs for BucketNotificationsHandler created by CDK stack (runtime Python):
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| timestamp | message |
|---------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1714068023786 | INIT_START Runtime Version: python:3.11.v34 Runtime Version ARN: arn:aws:lambda:us-east-2::runtime:1bebe65bb4f19dd8c37baeb0be18a0b278155b42bfe12dd28480199436d18229 |
| 1714068024198 | START RequestId: e5e39f6d-4059-4db8-96f5-a486f0c128cb Version: $LATEST |
| 1714068024959 | Status code: OK |
| 1714068024966 | END RequestId: e5e39f6d-4059-4db8-96f5-a486f0c128cb |
| 1714068024966 | REPORT RequestId: e5e39f6d-4059-4db8-96f5-a486f0c128cb Duration: 768.30 ms Billed Duration: 769 ms Memory Size: 128 MB Max Memory Used: 84 MB Init Duration: 409.95 ms |
| 1714068091179 | START RequestId: 85d47611-dfe2-4861-98f9-f3fa18e16e76 Version: $LATEST |
| 1714068091824 | Status code: OK |
| 1714068091830 | END RequestId: 85d47611-dfe2-4861-98f9-f3fa18e16e76 |
| 1714068091830 | REPORT RequestId: 85d47611-dfe2-4861-98f9-f3fa18e16e76 Duration: 651.07 ms Billed Duration: 652 ms Memory Size: 128 MB Max Memory Used: 85 MB |
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Thanks, Ashish
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
Thank you for your response Ashish.
- Typescript. The .py file comment is a reference to the python code used by our lamdba functions.
- Yes I am able to synthesize the stack. The error is thrown by CloudFormation during deployment.
- Those are the steps I follow to recreate the issue. I do have a lot of prefixes already defined, but whenever I add a new one the deployment stage throws an error. I resolve this issue by removing all existing prefixes and adding them all back along with the new target prefix
I'm having the same issue, with the below changes
Resources
[~] Custom::S3BucketNotifications imported_source_bucket/Notifications importedresourcesimportedsourcebucketNotifications2FD58448
└─ [~] NotificationConfiguration
└─ [~] .TopicConfigurations:
└─ @@ -55,6 +55,24 @@
[ ] },
[ ] {
[ ] "Events": [
[+] "s3:ObjectCreated:*"
[+] ],
[+] "Filter": {
[+] "Key": {
[+] "FilterRules": [
[+] {
[+] "Name": "suffix",
[+] "Value": ".mpd"
[+] }
[+] ]
[+] }
[+] },
[+] "TopicArn": {
[+] "Ref": "topicconstructshobjectcopyfanout2A14F786"
[+] }
[+] },
[+] {
[+] "Events": [
[ ] "s3:ObjectRemoved:*"
[ ] ],
[ ] "Filter": {
@@ -106,5 +124,23 @@
[ ] "TopicArn": {
[ ] "Ref": "topicconstructshobjectdeletefanout09E6ABCD"
[ ] }
[+] },
[+] {
[+] "Events": [
[+] "s3:ObjectRemoved:*"
[+] ],
[+] "Filter": {
[+] "Key": {
[+] "FilterRules": [
[+] {
[+] "Name": "suffix",
[+] "Value": ".mpd"
[+] }
[+] ]
[+] }
[+] },
[+] "TopicArn": {
[+] "Ref": "topicconstructshobjectdeletefanout09E6ABCD"
[+] }
[ ] }
[ ] ]
✨ Number of stacks with differences: 1
cdk version 2.139.1 (build b88f959)
stack is failing with the below:
9:14:26 PM | UPDATE_FAILED | Custom::S3BucketNotifications | importedresourcesi...ifications2FD58448
Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Co
nfiguration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. See the details in CloudWatch Log Stream:
2024/04/30/[$LATEST]e4ef3d9576c34417988f05f45f62285a (RequestId: 96d9d9dc-b8f7-47f5-ac33-02d3df762f43)
9:14:30 PM | UPDATE_FAILED | Custom::S3BucketNotifications | importedresourcesi...ifications2FD58448
Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Co
nfiguration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. See the details in CloudWatch Log Stream:
2024/04/30/[$LATEST]e4ef3d9576c34417988f05f45f62285a (RequestId: a63be1f4-7d52-42fa-a3f9-0dbfa2155d67)
❌ sns-fan-out-stack (sns-fan-out-stack) failed: Error: The stack named sns-fan-out-stack failed to deploy: UPDATE_ROLLBACK_FAILED (The following resource(s) failed to update: [importedresourcesimportedsourcebucketNotifications2FD58448]. ): Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. See the details in CloudWatch Log Stream: 2024/04/30/[$LATEST]e4ef3d9576c34417988f05f45f62285a (RequestId: 96d9d9dc-b8f7-47f5-ac33-02d3df762f43)
at FullCloudFormationDeployment.monitorDeployment (/opt/homebrew/lib/node_modules/aws-cdk/lib/index.js:433:10568)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/opt/homebrew/lib/node_modules/aws-cdk/lib/index.js:436:199619)
at async /opt/homebrew/lib/node_modules/aws-cdk/lib/index.js:436:181341
❌ Deployment failed: Error: The stack named sns-fan-out-stack failed to deploy: UPDATE_ROLLBACK_FAILED (The following resource(s) failed to update: [importedresourcesimportedsourcebucketNotifications2FD58448]. ): Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. See the details in CloudWatch Log Stream: 2024/04/30/[$LATEST]e4ef3d9576c34417988f05f45f62285a (RequestId: 96d9d9dc-b8f7-47f5-ac33-02d3df762f43)
at FullCloudFormationDeployment.monitorDeployment (/opt/homebrew/lib/node_modules/aws-cdk/lib/index.js:433:10568)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/opt/homebrew/lib/node_modules/aws-cdk/lib/index.js:436:199619)
at async /opt/homebrew/lib/node_modules/aws-cdk/lib/index.js:436:181341
The stack named sns-fan-out-stack failed to deploy: UPDATE_ROLLBACK_FAILED (The following resource(s) failed to update: [importedresourcesimportedsourcebucketNotifications2FD58448]. ): Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. See the details in CloudWatch Log Stream: 2024/04/30/[$LATEST]e4ef3d9576c34417988f05f45f62285a (RequestId: 96d9d9dc-b8f7-47f5-ac33-02d3df762f43)
@airmonitor Thank you for the detailed description. Can confirm I am getting the same error.
Worst part is that the stack is left in UPDATE_ROLLBACK_FAILED state, which requires manual intervention (skip updating the S3 bucket) for the rollback to complete.
@ashishdhingra any updates? This is blocking some of our critical deployments.
@thenamanpatwari As mentioned in the https://github.com/aws/aws-cdk/issues/29951#issuecomment-2077820432, I'm unable to reproduce the issue. Please share the sample end-to-end code (including Python code for your lambda function) and any other configuration that needs to be done, to reproduce the issue. From the error in the issue description, it appears that you have overlapping prefixes for the same event type. You also mentioned that there are lot of prefixes already defined. Are these defined for same event type?
Also the error in the issue description appears to be in Lambda event handler, looks like it is reported when the Lambda is invoked per event notification, not when CDK stack is deployed. Kindly correct if I'm wrong.
Kindly note that once the CDK deployment is initiated, it is CloudFormation that takes over.
Thanks, Ashish
Hello @ashishdhingra
I think you missed one variation in your tests.
To use the same prefix but for different actions.
prefix .jpg for s3:ObjectCreated:* prefix .jpg for s3:ObjectRemoved:*
this is failing.
Are you able to replicate now the issue or this is still not clear?
Regards
Hello @ashishdhingra
I think you missed one variation in your tests.
To use the same prefix but for different actions.
prefix .jpg for s3:ObjectCreated:* prefix .jpg for s3:ObjectRemoved:*
this is failing.
Are you able to replicate now the issue or this is still not clear?
Regards
@airmonitor Unable to reproduce using below code: TypeScript code
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import * as s3 from 'aws-cdk-lib/aws-s3';
import * as lambda from 'aws-cdk-lib/aws-lambda';
import * as s3n from 'aws-cdk-lib/aws-s3-notifications';
export class TypescriptStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const lambdaFunction = new lambda.Function(this, 'TestLambdaFunction',{
runtime: lambda.Runtime.NODEJS_18_X,
code: lambda.Code.fromAsset('lambda'),
handler: 'hello.handler'
});
const s3Bucket = new s3.Bucket(this, 'TestBucket');
s3Bucket.addEventNotification(s3.EventType.OBJECT_CREATED, new s3n.LambdaDestination(lambdaFunction), { prefix: ".jpg" });
s3Bucket.addEventNotification(s3.EventType.OBJECT_REMOVED, new s3n.LambdaDestination(lambdaFunction), { prefix: ".jpg" });
}
}
Output of cdk synth:
Resources:
TestLambdaFunctionServiceRole0C9E0634:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: "2012-10-17"
ManagedPolicyArns:
- Fn::Join:
- ""
- - "arn:"
- Ref: AWS::Partition
- :iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Metadata:
aws:cdk:path: TypescriptStack/TestLambdaFunction/ServiceRole/Resource
TestLambdaFunctionC089708A:
Type: AWS::Lambda::Function
Properties:
Code:
S3Bucket:
Fn::Sub: cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}
S3Key: 3e4845b15c28e9e3882e7f9016a42054e94d051bd8662ff76e8e9f9d0ae50c48.zip
Handler: hello.handler
Role:
Fn::GetAtt:
- TestLambdaFunctionServiceRole0C9E0634
- Arn
Runtime: nodejs18.x
DependsOn:
- TestLambdaFunctionServiceRole0C9E0634
Metadata:
aws:cdk:path: TypescriptStack/TestLambdaFunction/Resource
aws:asset:path: asset.3e4845b15c28e9e3882e7f9016a42054e94d051bd8662ff76e8e9f9d0ae50c48
aws:asset:is-bundled: false
aws:asset:property: Code
TestBucket560B80BC:
Type: AWS::S3::Bucket
UpdateReplacePolicy: Retain
DeletionPolicy: Retain
Metadata:
aws:cdk:path: TypescriptStack/TestBucket/Resource
TestBucketNotificationsE54C04C2:
Type: Custom::S3BucketNotifications
Properties:
ServiceToken:
Fn::GetAtt:
- BucketNotificationsHandler050a0587b7544547bf325f094a3db8347ECC3691
- Arn
BucketName:
Ref: TestBucket560B80BC
NotificationConfiguration:
LambdaFunctionConfigurations:
- Events:
- s3:ObjectCreated:*
Filter:
Key:
FilterRules:
- Name: prefix
Value: .jpg
LambdaFunctionArn:
Fn::GetAtt:
- TestLambdaFunctionC089708A
- Arn
- Events:
- s3:ObjectRemoved:*
Filter:
Key:
FilterRules:
- Name: prefix
Value: .jpg
LambdaFunctionArn:
Fn::GetAtt:
- TestLambdaFunctionC089708A
- Arn
Managed: true
DependsOn:
- TestBucketAllowBucketNotificationsToTypescriptStackTestLambdaFunction61C6B12C7EBA2523
Metadata:
aws:cdk:path: TypescriptStack/TestBucket/Notifications/Resource
TestBucketAllowBucketNotificationsToTypescriptStackTestLambdaFunction61C6B12C7EBA2523:
Type: AWS::Lambda::Permission
Properties:
Action: lambda:InvokeFunction
FunctionName:
Fn::GetAtt:
- TestLambdaFunctionC089708A
- Arn
Principal: s3.amazonaws.com
SourceAccount:
Ref: AWS::AccountId
SourceArn:
Fn::GetAtt:
- TestBucket560B80BC
- Arn
Metadata:
aws:cdk:path: TypescriptStack/TestBucket/AllowBucketNotificationsToTypescriptStackTestLambdaFunction61C6B12C
BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC:
Type: AWS::IAM::Role
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: "2012-10-17"
ManagedPolicyArns:
- Fn::Join:
- ""
- - "arn:"
- Ref: AWS::Partition
- :iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
Metadata:
aws:cdk:path: TypescriptStack/BucketNotificationsHandler050a0587b7544547bf325f094a3db834/Role/Resource
BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36:
Type: AWS::IAM::Policy
Properties:
PolicyDocument:
Statement:
- Action: s3:PutBucketNotification
Effect: Allow
Resource: "*"
Version: "2012-10-17"
PolicyName: BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36
Roles:
- Ref: BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC
Metadata:
aws:cdk:path: TypescriptStack/BucketNotificationsHandler050a0587b7544547bf325f094a3db834/Role/DefaultPolicy/Resource
BucketNotificationsHandler050a0587b7544547bf325f094a3db8347ECC3691:
Type: AWS::Lambda::Function
Properties:
Description: AWS CloudFormation handler for "Custom::S3BucketNotifications" resources (@aws-cdk/aws-s3)
Code:
ZipFile: |
import boto3 # type: ignore
import json
import logging
import urllib.request
s3 = boto3.client("s3")
EVENTBRIDGE_CONFIGURATION = 'EventBridgeConfiguration'
CONFIGURATION_TYPES = ["TopicConfigurations", "QueueConfigurations", "LambdaFunctionConfigurations"]
def handler(event: dict, context):
response_status = "SUCCESS"
error_message = ""
try:
props = event["ResourceProperties"]
notification_configuration = props["NotificationConfiguration"]
managed = props.get('Managed', 'true').lower() == 'true'
stack_id = event['StackId']
old = event.get("OldResourceProperties", {}).get("NotificationConfiguration", {})
if managed:
config = handle_managed(event["RequestType"], notification_configuration)
else:
config = handle_unmanaged(props["BucketName"], stack_id, event["RequestType"], notification_configuration, old)
s3.put_bucket_notification_configuration(Bucket=props["BucketName"], NotificationConfiguration=config)
except Exception as e:
logging.exception("Failed to put bucket notification configuration")
response_status = "FAILED"
error_message = f"Error: {str(e)}. "
finally:
submit_response(event, context, response_status, error_message)
def handle_managed(request_type, notification_configuration):
if request_type == 'Delete':
return {}
return notification_configuration
def handle_unmanaged(bucket, stack_id, request_type, notification_configuration, old):
def with_id(n):
n['Id'] = f"{stack_id}-{hash(json.dumps(n, sort_keys=True))}"
return n
external_notifications = {}
existing_notifications = s3.get_bucket_notification_configuration(Bucket=bucket)
for t in CONFIGURATION_TYPES:
if request_type == 'Update':
ids = [with_id(n) for n in old.get(t, [])]
old_incoming_ids = [n['Id'] for n in ids]
external_notifications[t] = [n for n in existing_notifications.get(t, []) if not n['Id'] in old_incoming_ids]
elif request_type == 'Create':
external_notifications[t] = [n for n in existing_notifications.get(t, [])]
if EVENTBRIDGE_CONFIGURATION in existing_notifications:
external_notifications[EVENTBRIDGE_CONFIGURATION] = existing_notifications[EVENTBRIDGE_CONFIGURATION]
if request_type == 'Delete':
return external_notifications
notifications = {}
for t in CONFIGURATION_TYPES:
external = external_notifications.get(t, [])
incoming = [with_id(n) for n in notification_configuration.get(t, [])]
notifications[t] = external + incoming
if EVENTBRIDGE_CONFIGURATION in notification_configuration:
notifications[EVENTBRIDGE_CONFIGURATION] = notification_configuration[EVENTBRIDGE_CONFIGURATION]
elif EVENTBRIDGE_CONFIGURATION in external_notifications:
notifications[EVENTBRIDGE_CONFIGURATION] = external_notifications[EVENTBRIDGE_CONFIGURATION]
return notifications
def submit_response(event: dict, context, response_status: str, error_message: str):
response_body = json.dumps(
{
"Status": response_status,
"Reason": f"{error_message}See the details in CloudWatch Log Stream: {context.log_stream_name}",
"PhysicalResourceId": event.get("PhysicalResourceId") or event["LogicalResourceId"],
"StackId": event["StackId"],
"RequestId": event["RequestId"],
"LogicalResourceId": event["LogicalResourceId"],
"NoEcho": False,
}
).encode("utf-8")
headers = {"content-type": "", "content-length": str(len(response_body))}
try:
req = urllib.request.Request(url=event["ResponseURL"], headers=headers, data=response_body, method="PUT")
with urllib.request.urlopen(req) as response:
print(response.read().decode("utf-8"))
print("Status code: " + response.reason)
except Exception as e:
print("send(..) failed executing request.urlopen(..): " + str(e))
Handler: index.handler
Role:
Fn::GetAtt:
- BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC
- Arn
Runtime: python3.11
Timeout: 300
DependsOn:
- BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleDefaultPolicy2CF63D36
- BucketNotificationsHandler050a0587b7544547bf325f094a3db834RoleB6FB88EC
Metadata:
aws:cdk:path: TypescriptStack/BucketNotificationsHandler050a0587b7544547bf325f094a3db834/Resource
CDKMetadata:
Type: AWS::CDK::Metadata
Properties:
Analytics: v2:deflate64:H4sIAAAAAAAA/02OywqDQAxFv6X7MdW66bYKXYt+QIljWuIjA2aslMF/L44UujoH7r0kF8jyK6QnXDWx3ZCM3EJoPNrB4KqPMOLUdgjhvoj17MSUT/n3iuaJVdnJZhgnCLUbaQ8iKzey/cRetM1o/kBV8gq3HUZzCMViB/IFKplD98Fh22ZqUrfMlkwcNB5fLK944ReUTymddOzjF+I6gl7P70sKWQbZqVfmZF7E80RQH/wCAE0K+fcAAAA=
Metadata:
aws:cdk:path: TypescriptStack/CDKMetadata/Default
Condition: CDKMetadataAvailable
Conditions:
CDKMetadataAvailable:
Fn::Or:
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- af-south-1
- Fn::Equals:
- Ref: AWS::Region
- ap-east-1
- Fn::Equals:
- Ref: AWS::Region
- ap-northeast-1
- Fn::Equals:
- Ref: AWS::Region
- ap-northeast-2
- Fn::Equals:
- Ref: AWS::Region
- ap-south-1
- Fn::Equals:
- Ref: AWS::Region
- ap-southeast-1
- Fn::Equals:
- Ref: AWS::Region
- ap-southeast-2
- Fn::Equals:
- Ref: AWS::Region
- ca-central-1
- Fn::Equals:
- Ref: AWS::Region
- cn-north-1
- Fn::Equals:
- Ref: AWS::Region
- cn-northwest-1
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- eu-central-1
- Fn::Equals:
- Ref: AWS::Region
- eu-north-1
- Fn::Equals:
- Ref: AWS::Region
- eu-south-1
- Fn::Equals:
- Ref: AWS::Region
- eu-west-1
- Fn::Equals:
- Ref: AWS::Region
- eu-west-2
- Fn::Equals:
- Ref: AWS::Region
- eu-west-3
- Fn::Equals:
- Ref: AWS::Region
- il-central-1
- Fn::Equals:
- Ref: AWS::Region
- me-central-1
- Fn::Equals:
- Ref: AWS::Region
- me-south-1
- Fn::Equals:
- Ref: AWS::Region
- sa-east-1
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- us-east-1
- Fn::Equals:
- Ref: AWS::Region
- us-east-2
- Fn::Equals:
- Ref: AWS::Region
- us-west-1
- Fn::Equals:
- Ref: AWS::Region
- us-west-2
Parameters:
BootstrapVersion:
Type: AWS::SSM::Parameter::Value<String>
Default: /cdk-bootstrap/hnb659fds/version
Description: Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]
Rules:
CheckBootstrapVersion:
Assertions:
- Assert:
Fn::Not:
- Fn::Contains:
- - "1"
- "2"
- "3"
- "4"
- "5"
- Ref: BootstrapVersion
AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI.
****************************************************
*** Newer version of CDK is available [2.142.0] ***
*** Upgrade recommended (npm install -g aws-cdk) ***
****************************************************
Output of cdk deploy:
✨ Synthesis time: 4.89s
TypescriptStack: start: Building 3d283875a592ba6e758163f993105c6a4a36103dcb9b43e2a92262295a53b354:current_account-current_region
TypescriptStack: success: Built 3d283875a592ba6e758163f993105c6a4a36103dcb9b43e2a92262295a53b354:current_account-current_region
TypescriptStack: start: Publishing 3d283875a592ba6e758163f993105c6a4a36103dcb9b43e2a92262295a53b354:current_account-current_region
TypescriptStack: success: Published 3d283875a592ba6e758163f993105c6a4a36103dcb9b43e2a92262295a53b354:current_account-current_region
This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:
IAM Statement Changes
┌───┬────────────────────────────────────────────────────────────┬────────┬──────────────────────────┬────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────┐
│ │ Resource │ Effect │ Action │ Principal │ Condition │
├───┼────────────────────────────────────────────────────────────┼────────┼──────────────────────────┼────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ + │ ${BucketNotificationsHandler050a0587b7544547bf325f094a3db8 │ Allow │ sts:AssumeRole │ Service:lambda.amazonaws.com │ │
│ │ 34/Role.Arn} │ │ │ │ │
├───┼────────────────────────────────────────────────────────────┼────────┼──────────────────────────┼────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ + │ ${TestLambdaFunction.Arn} │ Allow │ lambda:InvokeFunction │ Service:s3.amazonaws.com │ "ArnLike": { │
│ │ │ │ │ │ "AWS:SourceArn": "${TestBucket.Arn}" │
│ │ │ │ │ │ }, │
│ │ │ │ │ │ "StringEquals": { │
│ │ │ │ │ │ "AWS:SourceAccount": "${AWS::AccountId}" │
│ │ │ │ │ │ } │
├───┼────────────────────────────────────────────────────────────┼────────┼──────────────────────────┼────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ + │ ${TestLambdaFunction/ServiceRole.Arn} │ Allow │ sts:AssumeRole │ Service:lambda.amazonaws.com │ │
├───┼────────────────────────────────────────────────────────────┼────────┼──────────────────────────┼────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────┤
│ + │ * │ Allow │ s3:PutBucketNotification │ AWS:${BucketNotificationsHandler050a0587b7544547bf325f094a │ │
│ │ │ │ │ 3db834/Role} │ │
└───┴────────────────────────────────────────────────────────────┴────────┴──────────────────────────┴────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────┘
IAM Policy Changes
┌───┬────────────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────────────────────┐
│ │ Resource │ Managed Policy ARN │
├───┼────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${BucketNotificationsHandler050a0587b7544547bf325f094a3db834/Role} │ arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole │
├───┼────────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────────────────────┤
│ + │ ${TestLambdaFunction/ServiceRole} │ arn:${AWS::Partition}:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole │
└───┴────────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)
Do you wish to deploy these changes (y/n)? y
TypescriptStack: deploying... [1/1]
TypescriptStack: creating CloudFormation changeset...
✅ TypescriptStack
✨ Deployment time: 62.42s
Stack ARN:
arn:aws:cloudformation:us-east-2:139480602983:stack/TypescriptStack/e11f2ee0-13aa-11ef-8441-0a68c8ad7edb
✨ Total time: 67.31s
It also works if I use the equivalent Python code: Python code
from aws_cdk import (
# Duration,
Stack,
aws_s3 as s3,
aws_lambda as lam,
aws_s3_notifications as s3n
)
from constructs import Construct
class PythonStack(Stack):
def __init__(self, scope: Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs)
lambdaFunction = lam.Function(
self, "TestLambdaFunction",
runtime=lam.Runtime.NODEJS_18_X,
code=lam.Code.from_asset("lambda"),
handler="hello.handler"
)
s3Bucket = s3.Bucket(
self, "TestBucket"
)
s3Bucket.add_event_notification(s3.EventType.OBJECT_CREATED, s3n.LambdaDestination(lambdaFunction), s3.NotificationKeyFilter(prefix=".jpg"))
s3Bucket.add_event_notification(s3.EventType.OBJECT_REMOVED, s3n.LambdaDestination(lambdaFunction), s3.NotificationKeyFilter(prefix=".jpg"))
Output of cdk --version is 2.141.0 (build 3d1c06e)
Thanks, Ashish
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.
Still having the same issue
cdk --version 2.142.1 (build ed4e152)
Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Co
nfiguration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. See the details in CloudWatch Log Stream:
2024/05/21/[$LATEST]4bbc4c7e4dcc47de98ab1b4f1f7318c2 (RequestId: 86d05e5b-936c-4283-8c72-5584120eb8c2)
Traceback (most recent call last):
File "/var/task/index.py", line 24, in handler
s3.put_bucket_notification_configuration(Bucket=props["BucketName"], NotificationConfiguration=config)
File "/var/lang/lib/python3.11/site-packages/botocore/client.py", line 553, in _api_call
return self._make_api_call(operation_name, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lang/lib/python3.11/site-packages/botocore/client.py", line 1009, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.
Still having the same issue
cdk --version 2.142.1 (build ed4e152)
Received response status [FAILED] from custom resource. Message returned: Error: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Co nfiguration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.. See the details in CloudWatch Log Stream: 2024/05/21/[$LATEST]4bbc4c7e4dcc47de98ab1b4f1f7318c2 (RequestId: 86d05e5b-936c-4283-8c72-5584120eb8c2)Traceback (most recent call last): File "/var/task/index.py", line 24, in handler s3.put_bucket_notification_configuration(Bucket=props["BucketName"], NotificationConfiguration=config) File "/var/lang/lib/python3.11/site-packages/botocore/client.py", line 553, in _api_call return self._make_api_call(operation_name, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/var/lang/lib/python3.11/site-packages/botocore/client.py", line 1009, in _make_api_call raise error_class(parsed_response, operation_name) botocore.exceptions.ClientError: An error occurred (InvalidArgument) when calling the PutBucketNotificationConfiguration operation: Configuration is ambiguously defined. Cannot have overlapping suffixes in two rules if the prefixes are overlapping for the same event type.
@airmonitor Good afternoon. Could you please share self reproducible code sample to troubleshoot the issue? As per my previous comments, we are unable to reproduce the issue at our end, hence code sample from your side would be helpful.
Thanks, Ashish
Hello
I am wondering if you have covered a case when you are trying to add notification configuration to existing bucket which already has multiple events notifications configured.
The example which you showed above with your typescript code covers a happy path when you are creating new bucket and adding notification configuration to it.
So there is no existing configuration applied over there.
I feel that the lambda function which is adding new notification configuration it is trying to apply existing notification configuration to it by using overwrite Approach.
But in the reality that will not work because first you need to remove all of the notification configuration and apply the new one if old and new notification configurations have the same rules.
What do you think?
@airmonitor While running my example, I had first applied notification configuration for first prefix (commenting the code that adds bucket configuration for 2nd prefix). Thereafter, I uncommented the code that adds bucket configuration for 2nd prefix and deployed changes again.
It would be great if you could elaborate your scenario with step-by-step example.
Thanks, Ashish
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.
Comments on closed issues and PRs are hard for our team to see. If you need help, please open a new issue that references this one.
