Is there any callback function to handle TIMEOUT event?

[NatLee]

Hi!

I'm working on a project where I need to handle task timeout events and other unexpected errors. Is there any method that can execute a custom callback function when a task times out or encounters an error?

I've read the documentation, and the closest thing I found is the hook parameter. However, I've noticed that the hook is only called when a task completes normally. It seems that Django Q doesn't provide a method to handle cases where a task abnormally terminates, such as when it times out or encounters an error.

Here's what I'm trying to achieve:

def timeout_handler(task):
    print(f"Task {task.id} timed out after {task.timeout} seconds")
    # Custom logic to handle timeout
    # Update task status in database
    # Send notification
    # etc.

def error_handler(task):
    print(f"Task {task.id} encountered an error: {task.result}")
    # Custom logic to handle errors
    # Log the error
    # Update task status
    # Send alert
    # etc.

task_id = async_task(
    run_my_task, 
    arg1, 
    arg2, 
    timeout=3600,
    timeout_handler='path.to.timeout_handler',
    error_handler='path.to.error_handler'
)

Is there a way to implement this kind of functionality in Django Q? If not, what would be the recommended approach to handle these scenarios?

And, I've noticed that Python RQ (https://python-rq.org/docs/#job-callbacks) has similar functionality:

from rq import Callback
queue.enqueue(say_hello, 
              on_success=Callback(report_success),  # default callback timeout (60 seconds) 
              on_failure=Callback(report_failure, timeout=10), # 10 seconds timeout
              on_stopped=Callback(report_stopped, timeout="2m")) # 2 minute timeout  

Is it possible to implement something similar in Django Q? Having this kind of callback system would be extremely helpful for handling various task states, including timeouts and errors.

Thank you for your time and for maintaining this great package! :)

[GDay]

There isn't something like this right now, but I think this would be worthwhile to implement. Especially for cleaning up tasks.

I think it would be fairly straightforward to add this. We catch the timeout here, so we would have to trigger a new async_task from there to trigger the timeout event: https://github.com/django-q2/django-q2/blob/cb253357fbbc5489d69b9f1c31cbe8ed79808bcd/django_q/worker.py#L106-L107

Though a better setup would be to have a more advanced "guard" system that will take care of it, but that's a fairly big rewrite.

[NatLee]

Hi GDay!

Thank you for your quick response and for considering this feature.

I just added 2 simple hooks for handling timeout and failure exceptions.

https://github.com/django-q2/django-q2/compare/master...NatLee:django-q2:feat/add-timeout-and-failure-hooks

This implementation allows users to define custom functions to handle timeout and failure events, similar to the existing hook parameter for successful task completion.

Example usage:

def timeout_handler(task_dict):
    print(f"Task {task['id']} timeout.")
    # Custom timeout handling logic

def failure_handler(task_dict):
    print(f"Task {task['id']} failed with error.")
    # Custom failure handling logic

async_task(
    my_function,
    hook='path.to.success_handler',
    timeout_hook='path.to.timeout_handler',
    failure_hook='path.to.failure_handler'
)

One limitation of the current implementation is that the task passed to the handler functions is a dictionary rather than a Task object.

I think that this initial implementation might be a bit rough around the edges. It could potentially be more concise and elegant, but it works in my scenarios. :3

Thank you very much!🫡

[GDay]

Great work! I have made a PR for it and will probably make a few changes and then merge. Thanks a lot!!

[NatLee]

Thank you! I appreciate your quick response and action. I understand that adding this new feature to an older project that hasn't been fully refactored yet can be challenging, even though it's quite useful.

Thank you again for your work on this. 👍