Support DescribeExecution action in API Gateway

[jakezatecky]

This is a Feature Proposal

Description

Thanks for the plugin!

The serverless-step-functions plugin provides integration to the API Gateway by adding a hook to invoke the step function via the StartExecution action. When invoked, it returns the following output:

{
   "executionArn": "string",
   "startDate": number
}

This is not particularly useful if we want to use the step function as an API endpoint to wait for output. AWS supports a DescribeExecution event for step functions that, when the function has been completed, will provide output. Naturally, the executionArn has to be provided to invoke this action.

There are other ways to get output (the step function could theoretically call an endpoint to cause some type of web-socket communications), but short polling DescribeExecution seems like a common enough use case. It's pretty easy to create an integration to DescribeExecution via the API Gateway within the AWS console, but it does not appear to be supported easily in this plugin.

This might necessitate that the events configuration be changed a bit to distinguish between StartExecution and DescribeExecution. E.g.:

events:
   -
    http:
      path: status
      method: get
      cors: true
      action: DescribeExecution

Is this at all reasonable?

Additional Data

• Serverless Framework Core Version you're using: latest
• The Plugin Version you're using: latest
• Operating System: N/A
• Stack Trace: N/A
• Provider Error messages: N/A

[horike37]

Thank you for great proposal @jakezatecky :smile: I agree with the proposal and looks good adding the action syntax to 'events' section.

[ancil26]

+1 @jakezatecky This is exactly what we need for our system too, as we're using the front end to hit an API endpoint to retrieve a bunch of objects.

@horike37 can you point me to whereabouts in the source code that will need the necessary changes to allow this?

[horike37]

@ancil26 Great! Thank you for your effort! Although I don't exactly verify it, but it seems that it would change around here to specify DescribeExecution

Let us know if you have any other questions.

[ancil26]

Hi I changed that to DescribeExecution but it was erroring because executionArn was not being populated, it was null. I take it that I need to supply the executionArn somewhere here in the cloudformation template? Am i on the right track? If not any ideas where i can go from here

I'm having trouble trying to understand the general program flow. It's my first time working with serverless plugin btw

[IssueHuntBot]

@issuehuntfest has funded $20.00 to this issue. See it on IssueHunt

[edholland]

This feature would significantly increase the value of this integration!

[theburningmonk]

@horike37 Can API GW integration hit two endpoints sequentially? You can't describe the execution until you start it, and the API GW integration is proxying whatever user request to a startExecution request against Step Functions, but would need to make a second request to describeExecution.

I guess we can generate and slot a Lambda function in there, which can do this, but that seems overkill.

[edholland]

I dont think that there's any requirement to hit two endpoints sequentially within the framework - we just want the ability to be able to proxy HTTP requests to both GetExecution and StartExecution endpoints.

There is a blog post that describes the sort of thing we'd like to achieve (note: i'm not too bothered by hiding the ARN in the way they do here) https://medium.com/@cody_green/using-aws-api-gateway-and-step-functions-without-exposing-your-arn-ce94a88fa594

[theburningmonk]

I see, so you're not looking for ways to improve the response from the startExecution call, but for ways to add another endpoint that lets you get back the status/result of an execution (by passing in the execution ID somehow)?

[edholland]

Yes exactly, we want to startExecution and then poll a getExecution for eventually getting results.

[shota]

@edholland My patch for this issue was merged to master. https://github.com/horike37/serverless-step-functions#custom-step-functions-action

[edholland]

@shota Thank you greatly for this contribution, It's added significant value to this project!