django-postgres-queue
django-postgres-queue copied to clipboard
Mark jobs as failed persistently
After using this library in production for a while I have found that the current approach of keeping track of failed tasks in a set in memory doesn't work in a satisfactory way. I have multiple workers running jobs from the same queue. Every worker will pick up jobs that are bound to fail. Then, on a restart (i.e. deploy) the set is destroyed and the jobs are up for grabs again. So I thought... why not keep that state in the database on the Job objects themselves?
You can use the retry(0) decorator for this
You can use the
retry(0)decorator for this
I hadn't realized that. However, while that avoids running the job again it doesn't persist the state of failure and won't give me a chance to inspect the failed job, fix the error and possibly trigger it again.