Non-idempotent jobs (max_retries unset) that fail on a dead broker are not signalled

[bigjools]

Work was recently completed to send signals for jobs that fail as a result of a dead broker, however those jobs that have max_retries unset are not signalled as failed at all, unlike the idempotent ones which DO send a failure signal.

[some1ataplace]

Maybe try this.

Modify the Worker class in spinach to handle non-idempotent jobs that fail on a dead broker. Specifically, you can add a check in the process_job method to see if the job has max_retries unset, and if so, signal the job as failed if it fails due to a dead broker.

Here's an example implementation:

from spinach.worker import Worker

class CustomWorker(Worker):
    def process_job(self, job):
        try:
            self._process_job(job)
        except Exception as e:
            if not job.max_retries and 'dead' in str(e):
                self.send_failure_signal(job)
            else:
                raise e

[bigjools]

Fixed in 0.0.23