Allow tasks to have dependencies on task groups and vice versa

[bishopofbathandwells]

Currently, when we set a task to depend on a task group it doesn't work. Also, when setting a task group to depend on a task it doesn't work. It's not clear why either of these cases should be problematic. In fact, enjoy_breakfast.yml in the provided examples suggests their shouldn't be any issue.

Can anyone confirm whether there is some hidden complexity that I'm missing or if it should generally work as expected.

[bishopofbathandwells]

Some further context, a task dependency on a task group works if they are at the same level in the folder hierarchy. However, it doesn't work if the task is within its own task group. E.g. if you push enjoy_breakfast.yml into its own task group the dependencies on the groups are not recognised.

[chriscardillo]

@bishopofbathandwells thanks for sharing. I've got the following tasks:

task_a

operator: airflow.operators.bash.BashOperator
bash_command: date

my_group/task_b

operator: airflow.operators.bash.BashOperator
bash_command: date
dependencies:
  - task_a

Where task_b.yml lives inside of a folder called my_group.

The following graph is being rendered with no dependency issues.

using gusty==0.21.0 and airflow==2.8.0

Where are you running into this issue? Maybe the issue is being caused by suffixes and prefixes in the breakfast DAG?

[chriscardillo]

@bishopofbathandwells - I am going to close this issue for now because of inactivity. I apologize for not answering initially, as my notifications for this repository were not active.

You can have task groups depend on each other by specifying a dependencies block in the METADATA.yml of those task group folders.

I hope this helps.