kestra
kestra copied to clipboard
Add a `ResumeExecution` execution task to resume paused execution by ID
After pausing an execution with Pause task If you want to unpaused it, for now you can only handle it an api call or from the ui.
We should add Unpaused
task to do it directly from Kestra flow. By this way, we will be able to use for an example a trigger from a queue and to restart the execution based on the current message.
The design on the task can be tricky since :
- if the trigger is from the queue, we need to support internal storage
- if the trigger is from another task, we need to support raw data from outputs
Related discussion in slack
The motivation behind this feature is to offload async processing.
- Start a Kestra workflow execution
- The workflow gathers/preprocesses data
- The workflow sends the data for processing to an external system
- The workflow pauses itself (with a defined timeout)
- The external system sends an event signaling the processing finished (via a webhook or a MQ system)
- The workflow gets resumed
- Data received from the external system get evaluated and further processed
linked to https://github.com/kestra-io/kestra/issues/1581
Proposed syntax:
- id: pause
type: io.kestra.core.tasks.flows.Resume
executionId: "{{ execution.Id }}" # the default should be the current execution ID of this flow, but it can be any execution ID