Nike-Inc
[FEATURE] Add support for inter-workflow dependancy [parent-child workflows]
Databricks workflow now natively supports parent workflows to have child workflows.
Similar to Airflow - Dags and sub-dags.
refer - https://www.databricks.com/blog/modular-orchestration-databricks-workflows
Cloud Information
- [ ] AWS
- [ ] Azure
- [ ] GCP
- [ ] Other
Describe the solution you'd like User should be able to create workflow task like Notebook, Python, Shell task.
Describe alternatives you've considered Currently user needs to use WorkflowDependancySensor which is not the effective way.
Additional context Add any other context or screenshots about the feature request here.
@pariksheet, it sounds very much like RunJobTask that we already have + this pending enhancement https://github.com/Nike-Inc/brickflow/pull/227 Do you still want to keep it open?
