Paramjeet B comments

Results 40 comments of


                                            Paramjeet B

Tasks are in queued state for a longer time and executor slots are exhausted often

@jscheffl , Can you remove the pending response label.

Tasks are in queued state for a longer time and executor slots are exhausted often

After reviewing various GitHub and Stack Overflow discussions, I've made updates to the following configuration and migrated to `Airflow version 2.7.2` with `apache-airflow-providers-cncf-kubernetes version 8.0.0` : ``` [scheduler] task_queued_timeout :...

Tasks are in queued state for a longer time and executor slots are exhausted often

After observing for some time, we encountered instances where the executor open slots were approaching negative values, leading to tasks becoming stuck in the scheduled state. Restarting all the scheduler...

Tasks are in queued state for a longer time and executor slots are exhausted often

We have also observed that the pods are not cleaned up after completion of the task and all the pods are stuck in SUCCEEDED state.

Tasks are in queued state for a longer time and executor slots are exhausted often

Sorry , the above comment is false positive. We are customizing our KPO and we missed to add `on_finish_action` so the pods stuck in SUCCEEDED state. After adding it ,...

Tasks are in queued state for a longer time and executor slots are exhausted often

@dirrao , airflow num_runs configuration parameter purpose has been changed a while ago AFAIK and cannot be used for restarting the scheduler. We have also removed run_duration which was later...

Multiple GRPC errors while accessing the fsx pvc mount

@jacobwolfaws , Thanks for reverting back early. Is it possible mark this log as DEBUG than ERROR log.

Many running task instances are cleared by the new scheduler when an old scheduler is terminated and its health check server is periodically requested

@tanvn @RNHTTR , Could this issue be related to the one I raised : https://github.com/apache/airflow/issues/39096 We are running 8 schedulers and we see this issue happening often , where the...

Many running task instances are cleared by the new scheduler when an old scheduler is terminated and its health check server is periodically requested

> @dstandish > > ~I would like to build a Docker image from the source code (which I am going to add some modifications for testing) on my Macbook (Apple...

Airflow statsd stops sending metrics during maximum dagrun

@rawwar , yes I can confirm that the issue occurs intermittently while we run 250+ tasks in parallel