Dag is in Running state indefinitely But tasks were not getting triggered

[HemanthReddy2]

[levihernan]

I'm having the same issue, First run it goes OK, after the first one it just hangs and no task is ever run again.

Getting this log: Task is in the 'queued' state which is not a valid state for execution. The task must be cleared in order to be run.

[HemanthReddy2]

Hi,

For me this issue is because of my Cluster because of getting huge load on my workers so I have reduced max no of dag runs to 2/3 in parallel as I am having only 3 workers so that load can equally balanced. Earlier I use to run with 10-15 parallel runs so it got hanged in the state that I have mentioned. And limited the no. of tasks in a dag to max as 120-150 . So it's working fine now.

Hope this will help if the same kind of issue you are facing.

Regards Hemanth Reddy

On Fri, Sep 18, 2020, 9:25 PM Hernan Levi [email protected] wrote:

I'm having the same issue, First run it goes OK, after the first one it just hangs and no task is ever run again.

Getting this log: Task is in the 'queued' state which is not a valid state for execution. The task must be cleared in order to be run.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/puckel/docker-airflow/issues/580#issuecomment-694948265, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQOJRZ7BXS2ZIXYFMMBAI6LSGN7IJANCNFSM4PM3KTPQ .