airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Published 20 hours ago • verified publisher icon apache

Allow Task Group Ids to be passed as branches in BranchMixIn

Open boraberke opened this issue 10 months ago • 4 comments

This PR allows TaskGroup(s) to be passed as branch in the operators that use BranchMixIn.

With this implementation, if the provided string is a task_group_id instead of task_id, it will be expanded into the roots of that Task Group, as suggested here.

I would greatly appreciate any feedback and would love to work on it more!

Closes: https://github.com/apache/airflow/issues/34526

boraberke avatar Apr 09 '24 20:04 boraberke

As an additional question or note:

I am not sure if I should modify the docstring of the following operators:

  • BaseBranchOperator
  • BranchPythonOperator
  • BranchPythonVirtualenvOperator
  • BranchExternalPythonOperator
  • BranchDateTimeOperator
  • LatestOnlyOperator
  • BranchDayOfWeekOperator

They all mention single task_id or list of task_ids to follow. However, with this PR, they all start to support "single task_id, single task_group_id, and list of combination of task_ids and task_group_ids".

boraberke avatar Apr 09 '24 20:04 boraberke

Tests are failing. Can you fix them?

eladkal avatar Apr 14 '24 09:04 eladkal

There’s a test case needs fixing.

uranusjr avatar May 03 '24 06:05 uranusjr

As an additional question or note:

I am not sure if I should modify the docstring of the following operators:

  • BaseBranchOperator
  • BranchPythonOperator
  • BranchPythonVirtualenvOperator
  • BranchExternalPythonOperator
  • BranchDateTimeOperator
  • LatestOnlyOperator
  • BranchDayOfWeekOperator

They all mention single task_id or list of task_ids to follow. However, with this PR, they all start to support "single task_id, single task_group_id, and list of combination of task_ids and task_group_ids".

Lets please

eladkal avatar May 06 '24 08:05 eladkal