serverless-step-functions icon indicating copy to clipboard operation
serverless-step-functions copied to clipboard

is there any way to start the steps from a sns aws?

Open jeancarlosgarcia opened this issue 6 years ago • 15 comments

Are you planning to add to SNS events as serverless allows?

jeancarlosgarcia avatar Feb 23 '18 21:02 jeancarlosgarcia

Would love to do that if CloudFormation supports SNS trigger for StepFunctions.

In order to achieve that, you need to specify something on StepFunctions stuff to Protocol of AWS::SNS::Subscription. You can implement that if it is possible. https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sns-subscription.html#cfn-sns-protocol

horike37 avatar Feb 27 '18 16:02 horike37

I would love to see this too. Am I understanding the current limitation is that SNS doesn't support triggering a StepFunction yet?

image

I guess you could create a lambda function which just starts execution of the step function and then immediately executes. Seems sloppy though.

Just out of interest, if a lambda function calls a StepFunction asynchronously can it exit immediately and the StepFunction completes execution beyond the execution lifetime of the lambda function?

yankeeinlondon avatar Mar 02 '18 21:03 yankeeinlondon

Yes of course it can work perfectly, and so I have to do: I have a lambda that starts from an SNS event and that lambda invokes the StepFunction, the incidental lambda dies and the StepFunction takes its course.

jeancarlosgarcia avatar Mar 02 '18 21:03 jeancarlosgarcia

This is possible now. I came across this limitation recently and for now I'll use the above workaround but CloudWatch Events now supports Step Functions as a target. Looks like as of 3/21.

joshrussell avatar Mar 29 '18 09:03 joshrussell

Yes better to create a design like this

SNS Topic -> [event rule input] -> CloudWatch -> State machine

hasantayyar avatar Jun 21 '18 16:06 hasantayyar

@hasantayyar yes I heard CloudWatch supports State Machine as a target but all the examples are about the target. Can you explain CloudWatch Event Rule Pattern for SNS?

superhuman54 avatar Jul 20 '18 09:07 superhuman54

@superhuman54

screen shot 2018-07-20 at 12 45 43

this should be possible via CloudFormation But I believe you can not define the SNS topic here in the rule. so your state machine has to contain a logic and selectively continue or not based on context.


Update: I can't find a workaround with CloudFormation.

hasantayyar avatar Jul 20 '18 10:07 hasantayyar

I've tried to start StepFunction from CloudFront. This part works well if the rule is just a schedule one. If the rule is sns topic subscription, this doesn't work. I've failed to find event pattern working. Even the default one that should pick up all the events from sns without filtering fails.

FominArtmind avatar Aug 14 '18 07:08 FominArtmind

@hasantayyar @superhuman54 Have you got "SNS publications as CloudWatch Event source" (and thereby as a trigger for Step Functions) working? I just tried two hours to come up with a rule event pattern for this but there do not seem to be any events that can be filtered on - even when enabling cloud trail.

ploh avatar Aug 31 '18 15:08 ploh

@ploh I think that doesn't work as SNS in cloudwatch only track admin action as events. https://docs.aws.amazon.com/sns/latest/dg/sns-logging-using-cloudtrail.html

m4h3 avatar Jan 08 '19 16:01 m4h3

Any updates on this? Couldn't figure out if the sns -> step functions trigger is supported now or not just by looking at the documentation.

csabaszilveszter avatar May 18 '20 09:05 csabaszilveszter

Echoing your question @csabaszilveszter . Maybe there is still an AWS engineer out there listening. There seems to be a missing piece between the sns and the cloudWatch rule. I created an sns topic without any subscriptions and a cloudWatch rule that has an event source as in @hasantayyar screenshot above. The event do not have the option to point to a specific topic and a subscription on the topic do not have cloudWatch as an option in the protocol list. I manually published a message, and the step function was not invoked. I edited the topic and the step function was invoked. I created an s3 event on the bucket to publish put events to the topic, this too did not invoke the step function. @AWSengineers, this is my design, s3_bucket (put event) -> sns-topic ->??? cloudWatch Event -> step function. I know I can fire the event directly from s3 through cloudTrail and cloudWatch to invoke the step function but my supervisor wants all events to be published through sns. If you have a working example, please do point us to the part of the documentation. Thanks

zoadaare avatar Jun 05 '20 21:06 zoadaare

This doesn't seem to be possible. Going to SNS Console and creating a subscription for a topic, it does not show Step Functions as an option in the drop-down menu.

It would be a great addition to have this implemented: SNS --> Step Function

piersf avatar Jan 05 '21 17:01 piersf

One thing I'd like to point out limitations with the proposed work around using CloudWatch:

  • You can not ingest the payload of the published event.
  • You can't limit to just a publish API calls. So events like new subscription will trigger the pipeline.

mbrice1024 avatar Feb 26 '21 19:02 mbrice1024

Any updates?

I was trying now to raise an api gateway endpoint and subscribe to the sns topic with data. but sadly, I can't have my confirmation url :(

marcelinhov2 avatar Mar 08 '22 18:03 marcelinhov2