flyte
flyte copied to clipboard
[Docs] Overview of dynamic workflows from an execution POV
Description
The blog explains the dynamic workflow concept - https://blog.flyte.org/dynamic-workflows-in-flyte, but we need to add it to the documentation also. A few things that should be highlighted
- [ ] dynamic is still compiled - so it can be visualized
- [ ] understanding how futures file works
- [ ] When to use dynamic workflows, lower pressure on etcD
- [ ] problem of large fan-out, compare and contrast dynamic vs map tasks
- [ ] overriding resources and config at runtime dynamically using dynamic workflows
Are you sure this issue hasn't been raised already?
- [X] Yes
Have you read the Code of Conduct?
- [X] Yes
Fixes https://flyte-org.slack.com/archives/CNMKCU6FR/p1643105410130200?thread_ts=1642993031.095000&cid=CNMKCU6FR as well.