airflow icon indicating copy to clipboard operation
airflow copied to clipboard

Decorator for Task Flow (@skip_if, @run_if) to make it simple to apply whether or not to skip a Task -> Implementation also for Task Groups

Open weidinger-c opened this issue 1 year ago • 3 comments

Description

It would be really helpful, if the new feature @skip_if and @run_if would be available also for whole TaskGroups. With that I would be able to easily toggle the computation of whole TaskGroups with a simple check.

Use case/motivation

No response

Related issues

No response

Are you willing to submit a PR?

  • [ ] Yes I am willing to submit a PR!

Code of Conduct

weidinger-c avatar Oct 09 '24 12:10 weidinger-c

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

boring-cyborg[bot] avatar Oct 09 '24 12:10 boring-cyborg[bot]

This is currently not possible in Airflow 2 - because task group is mostly a grouping construct, not logical context that can be used as dependencies - the dependencies are between tasks. If you have the whole group with it's "entry"point" task - you can decorate this task with skip_if, or run-if and this is the way how you can do what you want I guess.

But maybe that's something that could be implemented after we release Airlfow 3.

I created a new label for such things airflow3.x candidate and marked it with it.

potiuk avatar Oct 12 '24 22:10 potiuk

Thanks for the reply, will test your suggestion by setting the first task of the task group with the skip_if flag.

weidinger-c avatar Oct 18 '24 12:10 weidinger-c