S3 Bucket Lambda notifications in Cloudformation for an Existing Bucket

[vemulagopal]

I am trying to add an S3 event to Lambda for an existing bucket using Cloudformation, but it is not working and says "S3 events must reference an S3 bucket in the same template". I checked few blogs but looks like this functionality is not supported via cloud formation. Would you please take it as a feature enhancement to allow the pre-existing s3 bucket as a trigger to Lambda using the Cloudformation.

Here is the CF script

LAMBDSample:
    Type: "AWS::Serverless::Function"
    Properties:
      FunctionName: LambdaSAMPLE
      CodeUri: "../lambda/lambda_function_Sample"
      Handler: "lambda_function.lambda_handler"
      MemorySize: 1200
      Role:
        Fn::GetAtt:
          - "ROLELAMBDASAMPLE"
          - "Arn"
      Runtime: "python3.6"
      Timeout: 900
      Events:
        SampleFileUpload:
          Type: S3
          Properties:
            Bucket: !FindInMap
              - AccountMap
              - !Ref "AWS::AccountId"
              - Bucket
            Events: s3:ObjectCreated:*
            Filter:
              S3Key:
                Rules:
                  - Name: prefix
                    Value: sample_files/
                  - Name: suffix
                    Value: .xml
      VpcConfig:
        SubnetIds:
          - !FindInMap
            - AccountMap
            - !Ref "AWS::AccountId"
            - PrivateSubnetAZa
          - !FindInMap
            - AccountMap
            - !Ref "AWS::AccountId"
            - PrivateSubnetAZb
        SecurityGroupIds:
          - !Ref SGLAMBDASAMPLE
    DependsOn:
      - SGLAMBDASAMPLE
      - ROLELAMBDASAMPLE

[keetonian]

This is similar to #124. I agree that this would be a nice feature; it would require either a change to how events work with the CFN S3 bucket resource, or using a custom resource to add the event to an existing resource.

[roelvandenbranddvc]

This seems familiar. We work with different stages of cloudformation in which buckets are created in a different CF than the Lambda functions. We created a workaround with a Lambda to fix a notification from S3 to SQS. This works for Lambdas as well if you change the NotificationConfiguration (CloudFunctionConfiguration) in the BucketConfiguration resource.

BucketConfiguration:
  Type: Custom::S3BucketConfiguration
  DependsOn:
  - ImportEventLambdaPermissionSortEventSQS
  - NotificationBucketPolicy
  Properties:
    ServiceToken: !GetAtt S3BucketConfiguration.Arn
    Bucket: {"Fn::ImportValue" : {"Fn::Sub" : "a-bucket-name"}}
    NotificationConfiguration:
      QueueConfigurations:
      - Events: ['s3:ObjectCreated:*']
        QueueArn: {"Fn::ImportValue" : {"Fn::Sub" : "a-sqs-arn"}}


S3BucketConfiguration:
    Type: AWS::Lambda::Function
    Properties:
        Description: Event S3 SQS Linker
        Runtime: nodejs8.10
        MemorySize: 128
        Timeout: 30
        Role: !GetAtt LambdaExecutionRole.Arn
        Handler: index.handler
        Code:
          ZipFile: !Sub |
            var response = require('cfn-response');
            var AWS = require('aws-sdk');
            var s3 = new AWS.S3();
            exports.handler = function(event, context) {
              var respond = (e) => response.send(event, context, e ? response.FAILED : response.SUCCESS, e ? e : {});
              process.on('uncaughtException', e=>failed(e));
              var params = event.ResourceProperties;
              delete params.ServiceToken;
              if (event.RequestType === 'Delete') {
                params.NotificationConfiguration = {};
                s3.putBucketNotificationConfiguration(params).promise()
                  .then((data)=>respond())
                  .catch((e)=>respond());
              } else {
                s3.putBucketNotificationConfiguration(params).promise()
                  .then((data)=>respond())
                  .catch((e)=>respond(e));
              }
            };

[keetonian]

https://github.com/aws-cloudformation/aws-cloudformation-coverage-roadmap/issues/79

[pam81]

+1

[roelvandenbranddvc]

I think i need to update the workaround in nodejs 10.x

the BucketConfiguration is still the same, updated the runtime and "cfn-response" is still supported. (seem to have lost the indentation required for yml)

S3BucketConfiguration: Type: AWS::Lambda::Function Properties: Description: PLI DGI Orchestrator S3 Event to SQS Linker Lambda Runtime: nodejs10.x MemorySize: 128 Timeout: 30 Role: !GetAtt MyRole.Arn Handler: index.handler Code: ZipFile: !Sub | var response = require('./cfn-response'); var AWS = require('aws-sdk'); var s3 = new AWS.S3(); exports.handler = function(event, context) { var respond = (e) => response.send(event, context, e ? response.FAILED : response.SUCCESS, e ? e : {}); process.on('uncaughtException', e=>failed(e)); var params = event.ResourceProperties; delete params.ServiceToken; if (event.RequestType === 'Delete') { params.NotificationConfiguration = {}; s3.putBucketNotificationConfiguration(params).promise() .then((data)=>respond()) .catch((e)=>respond()); } else { s3.putBucketNotificationConfiguration(params).promise() .then((data)=>respond()) .catch((e)=>respond(e)); } };