amplify-cli icon indicating copy to clipboard operation
amplify-cli copied to clipboard

Step Functions

Open GenesisChris opened this issue 5 years ago • 40 comments

Is your feature request related to a problem? Please describe. Integrating Step Functions is not currently supported, but is something I find myself using

Describe the solution you'd like I would love it if Step Functions were to become officially supported to be provisioned and given access in the Amplify CLI.

Describe alternatives you've considered I'm currently manually integrating the CloudFormation behind the scenes, or alternately using Terraform as a completely separate side application which then gets called from Amplify Apps.

Additional context AWS Step Functions is a very powerful solution for a lot of backend processes that I have behind the apps I write. To be able to natively use it in Amplify would be such a big time saver. Looks like the CloudFormation itself for it is pretty easy, but using Lambdas in the state machine is something I'm still figuring out and I'm guessing is the more difficult implementation blocker to overcome.

GenesisChris avatar Jan 17 '20 19:01 GenesisChris

Transferring this feature request to the amplify-cli repo since it is a CLI feature.

Ashish-Nanda avatar Jan 20 '20 22:01 Ashish-Nanda

I also would like to have this as well (which is something supported on the serverless framework as a comparison). By now, we are using custom CF stacks to setup step functions

rfpedrosa avatar Jan 29 '20 11:01 rfpedrosa

Is there any work being done on this? Worth submitting ideas / PR?

andrestone avatar Feb 04 '20 19:02 andrestone

@andrestone Currently there isn’t work going on this. Would be great if you could submit a public RFC before working on a PR and get some comments from the core team and the community.

kaustavghosh06 avatar Feb 04 '20 19:02 kaustavghosh06

+1 for this. Need step functions for use with Amplify CLI.

mormsbee avatar Apr 20 '20 13:04 mormsbee

Can you give us information on your business use case? This will help us with future designs. For example "In my app I need to provide a checkout system, etc." Thanks

undefobj avatar Apr 20 '20 21:04 undefobj

CRM Application. Someone creates an issue, or is even just working an issue and changes a field on a Case... either scenario could kickoff a workflow. Emails might need to be sent, tasks created and assigned, application wide notifications, the sky is the limit. At this point we are just evaluating the product but it seems like it might be a good fit. So far we've been able to stick to using the amplify cli for everything and I was actually surprised to see that there was no support for step functions currently.

mormsbee avatar Apr 21 '20 02:04 mormsbee

I'm also interested in Step functions.

My workflow is essentially request comes in -> identity check needs to be done on customer -> request assigned to appropriate person to fulfill -> request fulfilled -> "deliverable" shipped digitally or physically as appropriate -> communicate to customer -> charge customer card -> after a week send NPS score request for feedback.

It's similar to e-commerce workflow: order comes in -> fraud check -> assign to appropriate fulfillment center -> ship the item -> charge customer.

hisham avatar Apr 25 '20 15:04 hisham

Can someone please share their workarounds to integrate step functions into an Amplify application? I just started learning about Amplify and would love to have a detailed guide on how that's done.

cschow-ucsd avatar Jul 10 '20 02:07 cschow-ucsd

@cschow-ucsd I make step-functions working by creating a "Custom CloudFormation stacks". Similar to this this but for using Step Functions as orchestration instead of SNS topics for monitoring

rfpedrosa avatar Jul 10 '20 09:07 rfpedrosa

@cschow-ucsd I make step-functions working by creating a "Custom CloudFormation stacks". Similar to this this but for using Step Functions as orchestration instead of SNS topics for monitoring

Thanks! I have it working following the docs and a couple of examples at Github: https://github.com/aws-samples/aws-etl-orchestrator/blob/master/cloudformation/step-functions-resources.yaml https://github.com/widdix/aws-step-functions-example/blob/master/template.yml

I'm curious when this would be released. There are some plans on supporting this, right? It was a pain in the... fingers... to make it work doing it manually (it cost me 3 complete days to do this without previous CloudFormation knowledge).

My use case is to schedule some processes to be run a couple of minutes after, to check some flags, and then act in consequence to that. I've done that using EventBridge rules but it has some absurd limitations that cannot scale well.

I think we can write the state machine definition ourselves when doing the generation of the templates, but it would be great to have the rest automated as is in the other implementations (common things like role generation, environments, outputs, parameters, etc).

cristhian-boc avatar Jul 17 '20 19:07 cristhian-boc

+1 for Step functions with Amplify, as well as SNS and SQS! Currently, I'm using custom CF for adding SNS, and SQS, and will do the same for a workflow, but would be natural to be able to add them all through CLI.

asmajlovicmars avatar Aug 03 '20 15:08 asmajlovicmars

Hi for to the recent commenters on this thread - could you give more details on your use case and the application you're trying to build? We're interested in the specific use case as it pertains to a mobile or web application as this will help use scope design in the future. We don't have a timeline but this will help investigate.

undefobj avatar Aug 04 '20 02:08 undefobj

Hi for to the recent commenters on this thread - could you give more details on your use case and the application you're trying to build? We're interested in the specific use case as it pertains to a mobile or web application as this will help use scope design in the future. We don't have a timeline but this will help investigate.

Hi @undefobj , In our case, the particular application is like a multi-sided platform, that schedules events (interviews) between candidates and organizations. During that process a workflow is supposed to move candidates' applications through different stages. In that process the system will send SMS messages, SNS notifications, update status and move applicants through different stages as the requirements are completed, or not.

The application will be served on different devices, but I wouldn't say that it's predominantly a mobile app that we're building.

Not sure if this gives enough information, and the idea about the type of application we're building?

In the past I was part of the team that built a mortgage advancing application, that used FileNet (later IBM) Visual Workflow, which in a similar fashion moved objects through a workflow based on multitude of conditions. So, I'm looking for a similar usage.

asmajlovicmars avatar Aug 04 '20 13:08 asmajlovicmars

In our case we use a state machine for AWS Account creation as part of an Account Vending Machine. We are working on an Angular frontend for this Account Vending Machine with Amplify.

I ended up using the AWS API Gateway calling the step functions API. I have build a custom REST API with amplify "Custom CloudFormation stacks" which calls the step functions API.

Building this custom REST API was a bit cumbersome as the Amplify documentation is not that expressive about how to do that. The most difficult part was to get de api endpoint automatically in the aws-exports.js file when I preform an amplify push .

I eventually added a rest api with the amplify cli amplify add api and replaced the generated cloudformation code with my custom cloudformation code. This way when I perform an amplify push the endpoint is automatically added to the "aws_cloud_logic_custom" entry in the aws-exports file. I don't think this is the intended way to go but it works and I haven't found another way yet.

Perhaps it's an idea that on adding a REST API with the amplify cli you could also choose to integrate with other AWS Services like Step Functions.

koweblomke avatar Aug 11 '20 13:08 koweblomke

+1 for step functions with amplify. Is there anyway to see if this feature is currently being looked at, or if it's included in a roadmap / feature update?

EDIT: Just saw @undefobj comment, our use case is to be able to schedule events. We are currently using Cloudwatch Event Rules but this is not very scaleable since it's limited to a quite low number of rules. (100 i think)

mansdahlstrom1 avatar Sep 02 '20 12:09 mansdahlstrom1

+1

idanlo avatar Sep 08 '20 09:09 idanlo

+1.. I need to build an order management system.

shrishty avatar Sep 23 '20 14:09 shrishty

+1

michaelzxu avatar Oct 23 '20 14:10 michaelzxu

+1 We have two use-cases:

  1. Synthetic "big" data generator
  2. Custom glue orchestration (in fact, being able to describe glue config via Amplify would also be awesome)

jimjoes avatar Oct 30 '20 07:10 jimjoes

+1 Any update on this feature? It will be really useful to have step functions integration with amplify.

Mayank611 avatar Nov 24 '20 06:11 Mayank611

+1

AustinZhu avatar Dec 03 '20 15:12 AustinZhu

Use case for us with Step Functions (and SNS/SQS).

When a user visits the site and signs up, we want to send them a welcome email or text, along with a reminder email or text with some further prompts after a certain amount of time if they haven't further engaged with the site. For the case where we want to send a reminder, Step Functions are very handy, b/c they can be used as a kind of "cron" to wait a given amount of time before processing further, i.e. before kicking off the process that sends a message. The SQS parts of the flow are useful for storing large amounts of messages, so that the process will scale.

The flow generally looks like: User sign-up -> DynamoDB Stream -> Lambda -> SQS -> Lambda -> Step Functions -> SQS -> Lambda -> SNS -> Email/Text to user

Currently, we have this workaround where set up and maintain this stack using the Serverless framework, and it's a pain to manage this stack in parallel to the front-end/API which are in Amplify. But we do use some of the output of the Amplify deployment (such as the DynamoDB stream ID) as inputs for the Serverless stack, which is brittle, especially when other devs want to stand up their own personal version of the stack. Ideally, we would want this stack to be a part of our Amplify CI/CD. Also, it is hard to test this stack in conjunction with front-end work. Admittedly, we are also kicking off the deploy of this stack manually, which is a pain, we just haven't had the time or resources to invest in a better way. So if there was a straightforward way to spin up this stack, it would be greatly preferred.

chrisl777 avatar Dec 10 '20 22:12 chrisl777

+1

blairtaylor avatar Dec 17 '20 13:12 blairtaylor

+1

Use case for Step Functions

We are trying to make machine learning as easy as possible for people using SQL through Amazon Athena. We want to provide a unified experience for data analysts to leverage SageMaker from our SQL Interface.

User flow: User writes a SQL query with the target column and then submits a model training job. SQL Query + Target Column -> Step Function -> [create athena query execution -> write query to S3 -> Start SageMaker Autopilot job -> package model -> deploy endpoint] -> user makes predictions with Athena ML.

So far, we have not built anything yet but our assumption is that a Lambda function can trigger our state machine.

OElesin avatar Dec 29 '20 21:12 OElesin

+1

mikerudge avatar Mar 10 '21 23:03 mikerudge

This would be something we'd love to have.

Heavy users of both Amplify and Step Functions - not as much in combination, unfortunately - as we always have to set up the Step Functions manually and then make sure we have the right permissions on the Lambdas.

jonmifsud avatar Mar 12 '21 19:03 jonmifsud

+1

two-pack avatar Apr 02 '21 23:04 two-pack

+1 for step functions with Amplify

nniirraavv avatar Jun 29 '21 12:06 nniirraavv

+1

nijatb avatar Jul 26 '21 19:07 nijatb