astro-sdk
astro-sdk copied to clipboard
Add example DAGs and documentation on traditional SDK operators
Context
Many experienced Airflow users don't want to use Task Flow API. They consider it syntax sugar and would instead use traditional operators. This feedback was given by a couple of end-users, including @manmeetkaur.
Although users can use some SDK features, such as LoadFile and ExportFile, as standard operators, our docs don't explain or illustrate this.
All the following currently exist in the same "operators" category: decorators, tasks and dynamic task templates: https://astro-sdk-python.readthedocs.io/en/stable/guides/operators.html
None of the existing Python SDK example DAGs illustrates how to use LoadFile, TransformFile and ExportFile as old-school Airflow operators.
Acceptance criteria
- [ ] Create example DAGs illustrating how to use LoadFile and ExportFile without the need for TaskFlowAPI
- [ ] Update the documentation, to have separate categories for the following:
- [ ] decorators
- [ ] traditional operators
- [ ] tasks
- [ ] functions
- [ ] Create an RST page on how to use LoadFile / Export File