media-insights-on-aws icon indicating copy to clipboard operation
media-insights-on-aws copied to clipboard

Generic lambda-backed async await Opertator type

Open aburkleaux-amazon opened this issue 4 years ago • 1 comments

Create a new lambda-backed operator type to make use of lambda wait for callback pattern.

New operator type, AWAIT, takes a single lambda function as input. The operator state machine will start the task and save the callback TaskToken to the control plane where is can be used later by calling:

save_task_token(TaskToken, WorkflowExecutionId, AssetId, StageName,  OperatorName)

When asyncronous processing is complete, the asyncronous process can call the control plane to get the task token to callback to the workflow:

get_task_token(WorkflowExecutionId, AssetId, StageName,  OperatorName) 

aburkleaux-amazon avatar Apr 30 '20 18:04 aburkleaux-amazon

For reference, I have a partial implementation of this pattern (used to implement a Waiting status on workflows here

This issue decribes the implementation: https://github.com/aburkleaux-amazon/aws-media-insights-engine/issues/35

I think the main difference in implementation for a generic operator would be that the context would be stored in the control plane on the stage/operator part of the workflow_execution object. Error handling considerations might also be different.

aburkleaux-amazon avatar May 01 '20 15:05 aburkleaux-amazon