serverless-application-model
serverless-application-model copied to clipboard
aws
S3 Event triggers not working
Hi, I am facing an issue where Event is not being created and associated with Lambda, although it is specified in SAM:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Example
Resources:
LogToWatch:
Type: AWS::Serverless::Function
Properties:
Handler: index.handler
Runtime: python3.6
Timeout: 300
Policies: AmazonS3ReadOnlyAccess
Events:
S3CreateObject:
Type: S3
Properties:
Bucket:
Ref: TargetBucket
Events: s3:ObjectCreated:Put
TargetBucket:
Type: AWS::S3::Bucket
Finished up actual code - can be found here to replicate issue: https://github.com/wieshka/additional-cloudfront-metrics-from-access-logs
@wieshka What do you mean by 'Event is not being created'?
Are you not seeing the AWS::Lambda::Permission or a missing NotificationConfiguration Property on the bucket?
I deployed the template you have above, uploaded a file to S3, and saw the LambdaFunction be triggered. Am I missing something?
If you use the S3 event in SAM, then you don't see any trigger in the Lambda configuration panel.
But the Lambda function is executed when you drop a file in the S3 bucket.
Used the provided template to reproduce the issue.
The Lambda permissions created by SAM looks like this:
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "LogToWatchS3CreateObjectPermission-F3WNVM1OUAMX",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "lambda:invokeFunction",
"Resource": "arn:aws:lambda:<region>:<account_id>:function:LogToWatch-1WQI7PUOH1EF",
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "<account_id>"
}
}
}
]
}
But Lambda expects this in order to show the trigger in the console:
{
"Version": "2012-10-17",
"Id": "default",
"Statement": [
{
"Sid": "lambda-a5c4fbbd-61fc-4b08-82b4-7dd593c7f65f",
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "lambda:InvokeFunction",
"Resource": "arn:aws:lambda:us-east-1:303769779339:function:test-LogToWatch-1WQI7PUOH1EF",
"Condition": {
"StringEquals": {
"AWS:SourceAccount": "303769779339"
},
"ArnLike": {
"AWS:SourceArn": "<S3 bucket Arn>"
}
}
}
]
}
It is missing this part:
"ArnLike": {
"AWS:SourceArn": "<S3 bucket Arn>"
}
In the cloudformation template, the AWS::Lambda::Permission resource after transformed looks like this:
"LogToWatchS3CreateObjectPermission": {
"Type": "AWS::Lambda::Permission",
"Properties": {
"Action": "lambda:invokeFunction",
"SourceAccount": {
"Ref": "AWS::AccountId"
},
"FunctionName": {
"Ref": "LogToWatch"
},
"Principal": "s3.amazonaws.com"
}
}
It is missing the SourceArn property. It should be something like this:
"LogToWatchS3CreateObjectPermission": {
"Type": "AWS::Lambda::Permission",
"Properties": {
"Action": "lambda:invokeFunction",
"SourceAccount": {
"Ref": "AWS::AccountId"
},
"FunctionName": {
"Ref": "LogToWatch"
},
"Principal": "s3.amazonaws.com",
"SourceArn": {
"Fn::GetAtt": "TargetBucket.Arn"
}
}
}
However the issue is that, currently SAM makes the permission not scoped to a specific bucket. If we are to fix it in SAM making it scoped to a specific bucket so that it can show up properly in Lambda console, this could potentially break existing customer who expects broader permissions.
Just wondering why this is closed? I'm seeing the same issue. The bucket event source does not show up in the console. It makes it very confusing to know what is going on.
If you use the S3 event in SAM, then you don't see any trigger in the Lambda configuration panel. But the Lambda function is executed when you drop a file in the S3 bucket.
I get this problem too, when I deploy app to the aws, I config the s3 trigger, but I don't see it on the aws console, I also tried the dynamodb or api gateway trigger and they can show on the aws console. I don't know why?
Another question is I can't set the bucket name when I config s3 event; I find the older doc config they can set the s3 bucket name. Is there something different between the old and now version ?
@gartlady could you provide any additional information? The S3 triggers still don't show up in the lambda console, but should still work. Are you saying that you can't see the trigger or that the trigger doesn't work?
The S3 triggers are working but they are not appearing in the lambda console.
i am experiencing similar issue with SQS events. using SAM to deploy lambda with SQS event the lambda receives messages from the queue but the trigger is not visible via AWS console.
However the issue is that, currently SAM makes the permission not scoped to a specific bucket. If we are to fix it in SAM making it scoped to a specific bucket so that it can show up properly in Lambda console, this could potentially break existing customer who expects broader permissions.
please note that when using the Events you are expecting SPECIFIC permissions. giving broader permissions is an issue not a feature. when creating a lambda using SAM and giving an S3 as a trigger i expect that only that S3 is able to trigger the lambda. giving broader permissions than that seems unsecured.
Experienced the same issue. I can see no development was done for one year. Do someone knows another way to configure the same configuration (without SAM)?
Experienced the same issue. I can see no development was done for one year. Do someone knows another way to configure the same configuration (without SAM)?
use: Type: AWS::Lambda::EventSourceMapping https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-eventsourcemapping.html
exampleTrigger:
Properties:
BatchSize: 1
Enabled: true
EventSourceArn:
Fn::GetAtt:
- source
- Arn
FunctionName:
Fn::GetAtt:
- CreatedLambda
- Arn
Type: AWS::Lambda::EventSourceMapping
Experienced the same issue. I can see no development was done for one year. Do someone knows another way to configure the same configuration (without SAM)?
use: Type: AWS::Lambda::EventSourceMapping https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-eventsourcemapping.html
exampleTrigger: Properties: BatchSize: 1 Enabled: true EventSourceArn: Fn::GetAtt: - source - Arn FunctionName: Fn::GetAtt: - CreatedLambda - Arn Type: AWS::Lambda::EventSourceMapping
I can see that for event source arn only next values are available
Amazon Kinesis – Default 100. Max 10,000. Amazon DynamoDB Streams – Default 100. Max 1,000. Amazon Simple Queue Service – Default 10. Max 10.
No S3 option.
This is a breaking change, so we wouldn't be able to do this unless we made a new version of SAM. I agree, though, that this should be fixed.
Lu's comment (https://github.com/awslabs/serverless-application-model/issues/300#issuecomment-408950770) describes what needs to be changed.
~Late to the party, but I think I have the problem with a particular Lambda not showing a Cloudwatch trigger in the console. My other Lambdas (with basically the same trigger) show them in the console.~
~Seems like the Function Policy went missing at some point, although after creating the lambda 2 years ago it was there.~
Actually it was because my trigger was attached to a particular ALIAS that it didn't show up in the console for the "root" Lambda. Switching the version dropdown made it come back. I noticed this when looking at the rules in Cloudwatch, as the rule shows its targets at the bottom of the console, and clicking the target will jump to the correct version.
Does everyone resolve this issue? I've got the same issue. The Cloud formation just only create the permission resource.
Still having the issue as well. The "fix" to specify the Bucket Arn in the Permission is not viable because it will create an implicit loop between the bucket and the permission: bucket needs the permission to create the notification configuration but the permission needs the bucket arn.
@phongtran227 @pierremarieB
You can add a AWS::Lambda::Permission to your Resources. It should look something like the following:
LambdaInvokePermission:
Type: 'AWS::Lambda::Permission'
Properties:
FunctionName: !GetAtt MyFunction.Arn
Action: 'lambda:InvokeFunction'
Principal: 's3.amazonaws.com'
SourceAccount: !Sub ${AWS::AccountId}
SourceArn: !GetAtt MyBucket.Arn
That worked for me.
@henrikbjorn I tried your suggestion but it doesn't show the S3 triggers on console. Pretty new to this so using docs and all to find my way. I using SAM CLI to bundle and upload packaged artifact to an S3 bucket. To test I simply pass the S3 URL of uploaded artifact to Lambda function, everything else shows up but no S3 trigger shown on console, no errors either.
@moosakhalid You also need access to the S3 bucket. I found that it is important to specify Version when specifying policies.
I have only tried where i created the s3 bucket in the same CloudFormation template.
@henrikbjorn thanks for the help and hints. I finally got my SAM deployment to work though thanks to your Lambda Permissions snippet! Your method worked because that's the only way SAM/CFN deployment will let you refer to an S3 bucket i.e. if it's created within same template. It is rather annoying but I guess it's there for a good reason perhaps.
Confirmed that @henrikbjorn Lambda Permissions snippet worked for us too. Thanks!
The @henrikbjorn Lambda Permission snippet wasn't working for me by itself, but then the observation by @moosakhalid that the resources had to be created in the same template triggered an idea (no pun).
Despite having @henrikbjorn snippet, when I tried to add the trigger manually, I got "The provided execution role does not have permissions to call ReceiveMessage on SQS (Service: AWSLambda". Ah ha.
My SQS queue is being created in a different template, and I'm using SAM managed policy SQSPoller in the function Policies statement to allow lambda to read the queue.
However, I note that when you click "Edit Permission Boundary" on the Lambda Application page, the dialog notes:
This policy statement includes ARNs for the resources created in your SAM template. If your application needs access to resources outside of the SAM template, add those resource ARNs to the statement.
Indeed. The Permission Boundary didn't include permissions granted by SQSPoller policy for the Lambda function to access the externally created resource that I am importing with ImportValue.
So in addition to @henrikbjorn snippet, I copied the SQSPoller permissions from the Lambda Execution Role into the PermissionsBoundaryPolicy, et voila.
Now my trigger shows in the console.
This leads me to conclude that the actual bug is the failure of SAM / CloudFormation to error with an explanation that permissions are incomplete. Whereas the Console does give a reasonably clear error that it thinks permissions are inadequate and won't add the trigger until they are fixed.
https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/79
If you use the S3 event in SAM, then you don't see any trigger in the Lambda configuration panel. But the Lambda function is executed when you drop a file in the S3 bucket.
This saved me a lot of time.. thank you
