metaflow
metaflow copied to clipboard
Implementation for triggering a SFN execution from the result of a previous SFN execution
An implementation to allow SFN Executions to be triggered from the finishing state of another SFN execution. I have been using this to trigger multiple consumer Flows when a specific Producer flow finishes. For example:
Hourly -> DataCleaningFlow DataCleaningFlow -> Analysis1 DataCleaningFlow -> Analysis2
This ensures that when Analysis1/2 run, they will be able to obtain the data from the DataCleaningFlow, no need to guess at when it might end.
To implement this, I've modified the @schedule annotation to take a flow_name. This could be extended to add multiple flows and wait for their state (currently only support SUCCEEDED from SFN).
Closes #837
Syntax still requires formatting.
@iiAnderson I am taking the next couple of weeks off. I can take a look at this once I am back. Would that work?
No rush @savingoyal, just a feature I've found useful!
Hi @savingoyal, any updates on this PR? This would be a really useful feature to have!