BUG: Job status is not updated when checked in `on_success` and `on_failure` callbacks

[caffeinatedMike]

Currently, it appears that job status is stuck at started when checked within an on_success or on_failure callback. Since the job can have different statuses (at least when it comes to on_failure) I think it makes sense to update the job status before the worker invokes the callbacks.

Code to reproduce

from redis import Redis
from rq import Queue
from app.compat import SimpleWorker

def dummy_func():
    raise ValueError("Job should be set to 'failed' after this")

def failure_callback(job, connection, *args, **kwargs):
    print("this will say 'started' ->", job.get_status())

def test():
    q = Queue(connection=Redis.from_url("redis://"))
    w = SimpleWorker([q], connection=q.connection)
    q.empty()
    job = q.enqueue("dummy_func", on_failure=failure_callback)
    w.work(burst=True)

[selwin]

Yes, what you said makes sense. The job's status should be changed to reflect it's latest status before being passed on to job callbacks.

[anhvut]

I would like to add that in case of success, the result is not saved in the job yet when calling the callback.

[shirecoding]

any updates on this? seems to still be the case

[ccrvlh]

I think the easiest solution here would be to set the status to FAILED right at the beginning of this except block

https://github.com/rq/rq/blob/eef6dbbb3cedf409209eaace0823dd220e0bde70/rq/worker.py#L1183-L1188

This is not optimal though, since the same operation happens again when going through the registries. I guess the main challenge here is that we are calling set_status from multiple different places. Wondering if we should change the status on the perform method of the job.

https://github.com/rq/rq/blob/eef6dbbb3cedf409209eaace0823dd220e0bde70/rq/job.py#L1168-L1181

I guess all of the execution goes through here, so it could be a first filter.

@selwin let me know what you prefer here.