airflow icon indicating copy to clipboard operation
airflow copied to clipboard

feat: soft_fail TriggerDagRunOperator

Open raphaelauv opened this issue 1 year ago • 0 comments

If I have a lot of TriggerDagRunOperator in failed stated ( where some failed before triggering the dag_run and some after ) for any reason ( airflow worker unclean stop .., zombie task ... )

then if I clear all of them , some will be running then success and others will directly fail because dag_run already exists and was correctly trigger

So it's painful to manage this operator at real scaling ( thousands of runs )

with this option we can clear the operator and it will not fail in case the dag_run was correctly trigger

raphaelauv avatar Apr 22 '24 16:04 raphaelauv