Have `BaseOperator` get DAG from current task group if necessary
If there isn't an active DAG context but there is an active task group context where the task group has an associated DAG, then currently operators won't get the DAG from the task group, so we either have to explicitly trigger a new DAG context:
# Assume `dag` has been defined elsewhere without using a DAG context.
with dag:
with TaskGroup(...):
task = Operator(...)
or manually pass the DAG into every operator:
# Assume `dag` has been defined elsewhere without using a DAG context.
with TaskGroup(..., dag=dag):
task = Operator(..., dag=dag)
This PR changes BaseOperator to get the DAG from the current task group if it hasn't been manually passed in or found via an active DAG context:
# Assume `dag` has been defined elsewhere without using a DAG context.
with TaskGroup(..., dag=dag):
task = Operator(...)
assert task.dag == dag
^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named {pr_number}.significant.rst or {issue_number}.significant.rst, in newsfragments.
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
This sounds reasonable to me. Although the dag=dag approach is generally disfavoured nowadays, and you can always do
with dag, TaskGroup(...):
...
instead, fetching the DAG from the TaskGroup context seems like the right thing to do.
The tests need to be fixed though.
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
@sean-rose can you fix the tests?
@sean-rose can you fix the tests?
Sure, I'll try to find some time soon to take a crack at that.
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.