Render Airflow TaskGroups with Clusters

[nathadfield]

Airflow DAGs that utilise TaskGroups would be better represented if we were to make use of Clusters

[feluelle]

The problem I have with this feature is that retrieving the information from airflow whether it belongs to a taskgroup or not is unsafe. We cannot retrieve it from the rest api directly as it currently only is a UI concept. I have to parse the task_id 😟.

But we can give it a try.

[nathadfield]

Yes, that's true and presumably gets trickier if there are multiple levels of TaskGroups? I know there is a desire to make TaskGroups more than just a UI construct so perhaps this will be easier in the future.

[feluelle]

Exactly. I believe I have heard something similar, too. Once it becomes a not-only UI concept anymore, tackling this issue will become much easier.