jets icon indicating copy to clipboard operation
jets copied to clipboard

Support for s3 event sns configuration

Open SteffanPerry opened this issue 10 months ago • 2 comments

This is a 🙋‍♂️ feature or enhancement.

  • [x] I've added tests (if it's a bug, feature or enhancement)
  • [ ] I've adjusted the documentation (if it's a feature or enhancement)
  • [x] The test suite passes (run bundle exec rspec to verify this)

Summary

Allow custom SNS properties, such as filter policies, to be applied to s3_events.

class HardJob
  S3_BUCKET = "test-bucket".freeze
  SUBSCRIPTION_PROPS  = {
    FilterPolicyScope: "MessageBody",
    FilterPolicy: {
      Records: {
        s3: {
          object: {
            key: [
              { prefix: "test-prefix/" }
            ]
          }
        }
      }
    }.to_json
  }.freeze
  
  # Example of a job that is triggered by an S3 event with a filter policy
  # This will only be triggered by items added to the "test-bucket" bucket in the "test-prefix" folder
  s3_event(S3_BUCKET, sns_subscription_properties: SUBSCRIPTION_PROPS)
  def filtered
    puts "Filtered event: #{JSON.dump(event)}"
  end

  # Example of a job that is triggered by an S3 event without a filter policy
  # This will be triggered by all items added to the "test-bucket" bucket
  s3_event(S3_BUCKET)
  def unfiltered
    puts "Unfiltered event: #{JSON.dump(event)}"
  end
end

Context

https://github.com/rubyonjets/jets/issues/534

How to Test

Test repo: https://github.com/SteffanPerry/s3-event-custom-properties

SteffanPerry avatar Apr 17 '24 17:04 SteffanPerry

@tongueroo Thoughts on this & naming convention?

SteffanPerry avatar Apr 19 '24 17:04 SteffanPerry

Think that sns_subscription_properties is fine.

tongueroo avatar Apr 19 '24 17:04 tongueroo

@tongueroo Ive created a PR for updated docs, I think we can merge this in.

SteffanPerry avatar May 16 '24 13:05 SteffanPerry