docker-airflow
docker-airflow copied to clipboard
Dag is in Running state indefinitely But tasks were not getting triggered
I'm having the same issue, First run it goes OK, after the first one it just hangs and no task is ever run again.
Getting this log: Task is in the 'queued' state which is not a valid state for execution. The task must be cleared in order to be run.
Hi,
For me this issue is because of my Cluster because of getting huge load on my workers so I have reduced max no of dag runs to 2/3 in parallel as I am having only 3 workers so that load can equally balanced. Earlier I use to run with 10-15 parallel runs so it got hanged in the state that I have mentioned. And limited the no. of tasks in a dag to max as 120-150 . So it's working fine now.
Hope this will help if the same kind of issue you are facing.
Regards Hemanth Reddy
On Fri, Sep 18, 2020, 9:25 PM Hernan Levi [email protected] wrote:
I'm having the same issue, First run it goes OK, after the first one it just hangs and no task is ever run again.
Getting this log: Task is in the 'queued' state which is not a valid state for execution. The task must be cleared in order to be run.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/puckel/docker-airflow/issues/580#issuecomment-694948265, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQOJRZ7BXS2ZIXYFMMBAI6LSGN7IJANCNFSM4PM3KTPQ .