Stuck jobs in queue

[Olegt0rr]

Question: Why some tasks queued but not going to execution? New task processed by workers immediately.

Healthcheck: Jun-20 10:35:06 j_complete=6 j_failed=1 j_retried=0 j_ongoing=1 queued=10

WorkerSettings: max_jobs = 10

Workers: 4 replicas CPU, RAM and drive space is OK.

Redis: 12 items - is OK, because 2 was added after health check.

[JonasKs]

Please look at #343, you have to also check if the arq:retry or arq:in-progress have the same GUID.

[Olegt0rr]

I increase job_timeout cause my job may work up to 3 days - it's okay

[JonasKs]

Then it is because of what I wrote in #343 and this is intended. 😊 I think this issue can be closed.

[Olegt0rr]

@JonasKs, so what should I do with long tasks? May be restart timeout and "auto-kill" timeout should be separate?

[JonasKs]

The only way would to build in a health check of some sort from the worker, as suggested in the other issue.

What you should do is honestly to split your tasks into multiple steps (tasks), or to use func to set only a high timeout on your long tasks, so all other tasks can have a more sensible timeout.

[samuelcolvin]

Good answer.