django-postgres-queue icon indicating copy to clipboard operation
django-postgres-queue copied to clipboard

Mark jobs as failed persistently

Open Majsvaffla opened this issue 2 years ago • 2 comments
trafficstars

After using this library in production for a while I have found that the current approach of keeping track of failed tasks in a set in memory doesn't work in a satisfactory way. I have multiple workers running jobs from the same queue. Every worker will pick up jobs that are bound to fail. Then, on a restart (i.e. deploy) the set is destroyed and the jobs are up for grabs again. So I thought... why not keep that state in the database on the Job objects themselves?

Majsvaffla avatar Nov 01 '23 15:11 Majsvaffla

You can use the retry(0) decorator for this

gavinwahl avatar Nov 08 '23 17:11 gavinwahl

You can use the retry(0) decorator for this

I hadn't realized that. However, while that avoids running the job again it doesn't persist the state of failure and won't give me a chance to inspect the failed job, fix the error and possibly trigger it again.

Majsvaffla avatar Jan 31 '24 13:01 Majsvaffla