S3 Events with EventBridge

[rpf3]

I have configured my S3 bucket to send events using EventBridge as described in this recently released feature. Below is an example JSON payload that a Lambda Function will recieve.

{
  "version": "0",
  "id": "17793124-05d4-b198-2fde-7ededc63b103",
  "detail-type": "Object Created",
  "source": "aws.s3",
  "account": "123456789012",
  "time": "2021-11-12T00:00:00Z",
  "region": "ca-central-1",
  "resources": ["arn:aws:s3:::example-bucket"],
  "detail": {
    "version": "0",
    "bucket": {
      "name": "example-bucket"
    },
    "object": {
      "key": "example-key",
      "size": 5,
      "etag": "b1946ac92492d2347c6235b4d2611184",
      "version-id": "IYV3p45BT0ac8hjHg1houSdS1a.Mro8e",
      "sequencer": "00617F08299329D189"
    },
    "request-id": "N4N7GDK58NMKJ12R",
    "requester": "123456789012",
    "source-ip-address": "1.2.3.4",
    "reason": "PutObject"
  }
}

I was looking into using the S3Events package but classes comprising the S3EventNotification model do not quite line up with this JSON object; specifically the nested Records array.

Am I overlooking something or does there need to be a different Lambda event model for this new event type?

[ashishdhingra]

Hi @rpf3,

Good morning.

Thanks for posting the guidance question. Upon investigation, looks like support for model classes for EventBridge is missing from Lambda repository. It is no where related to Amazon.Lambda.S3Events package. Refer Using AWS Lambda with Amazon EventBridge (CloudWatch Events) which provides example EventBridge (CloudWatch Events) event. Kindly note that the source and other details might change. Amazon EventBridge events provides some details on the schema of an EventBridge event. EventBridge not only works with S3, but also works with other AWS services each of which could have a different detail (refer Events from AWS services).

As a workaround, you could develop your custom POCO (model) classes per your use case and use it to receive Lambda event.

Hope this provides necessary guidance.

Thanks, Ashish

[rpf3]

Thanks for the quick reply @ashishdhingra. If I develop some custom POCO classes for this use case would that be a good fit for a PR back into the Amazon.Lambda.CloudWatchEvents package?

[ashishdhingra]

Thanks for the quick reply @ashishdhingra. If I develop some custom POCO classes for this use case would that be a good fit for a PR back into the Amazon.Lambda.CloudWatchEvents package?

@rpf3 That appears to be the case if I look at https://github.com/aws/aws-lambda-dotnet/blob/master/Libraries/src/Amazon.Lambda.CloudWatchEvents/CloudWatchEvent.cs. You would need to follow the convention as mentioned in Readme for S3 service. Please feel free to submit a PR which could be reviewed by the team. I would convert this issue to feature-request.

[rpf3]

I've pushed an initial PR just to get some feedback on the overall approach. I'm having issues getting the project to build locally and run the unit tests. Is there a more comprehensive developer guide other than the Contributing.md file? Maybe something with development environment requirements and how to build/run tests?

[ashishdhingra]

I've pushed an initial PR just to get some feedback on the overall approach. I'm having issues getting the project to build locally and run the unit tests. Is there a more comprehensive developer guide other than the Contributing.md file? Maybe something with development environment requirements and how to build/run tests?

@rpf3 Simply cloning the repository and building should work. Not sure about the issues on your machine. Are there any specific errors you are getting?

[ashishdhingra]

@rpf3 Please review the last in the PR.

[github-actions[bot]]

This issue has not received a response in 5 days. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.

[rpf3]

@ashishdhingra in addition to a few hundred warnings, when I clone and run a dotnet build I get the following errors.

/home/rpf3/src/aws-lambda-dotnet/Libraries/test/Amazon.Lambda.RuntimeSupport.Tests/Amazon.Lambda.RuntimeSupport.IntegrationTests/Amazon.Lambda.RuntimeSupport.IntegrationTests.csproj(42,5): error MSB3073: The command "dotnet lambda package -c Release" exited with code 255.
/home/rpf3/src/aws-lambda-dotnet/Libraries/test/EventsTests.Shared/EventTests.cs(1903,53): error CS1503: Argument 2: cannot convert from 'int' to 'System.Collections.Generic.IEnumerable<char>' [/home/rpf3/src/aws-lambda-dotnet/Libraries/test/EventsTests.NETCore31/EventsTests.NETCore31.csproj]
/home/rpf3/src/aws-lambda-dotnet/Libraries/test/EventsTests.Shared/EventTests.cs(1903,53): error CS1503: Argument 2: cannot convert from 'int' to 'System.Collections.Generic.IEnumerable<char>' [/home/rpf3/src/aws-lambda-dotnet/Libraries/test/EventsTests.NET6/EventsTests.NET6.csproj]
    924 Warning(s)
    3 Error(s)

Time Elapsed 00:01:36.96

I am running dotnet version 6.0.202. I don't see anything documented about supported versions but these errors prevent me from running the full test suite.

[rpf3]

Actually I just pulled the latest from master and it now builds without issue.

[normj]

The PR with this change has been released as part of version 4.1.0 of Amazon.Lambda.CloudWatchEvents.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.