django-background-tasks icon indicating copy to clipboard operation
django-background-tasks copied to clipboard

MAX_RUN_TIME not respected when BACKGROUND_TASK_RUN_ASYNC is True

Open camuthig opened this issue 5 years ago • 4 comments

When running with BACKGROUND_TASK_RUN_ASYNC=True, the MAX_RUN_TIME is not respected. I believe this is happening because the locked() queryset is checking for tasks that either have a locked_by value or has a locked_at greater than the expiration time. As we know that the task is locked, it will definitely have a locked_by value, and will always return true. When you combine this with the below logic in find_available, if you have BACKGROUND_TASK_ASYNC_THREADS number of tasks running over MAX_RUN_TIME, then all of them will be counted as part of the locked count, and therefore ready is set to self.none()

            currently_failed = self.failed().count()
            currently_locked = self.locked(now).count()
            count = app_settings.BACKGROUND_TASK_ASYNC_THREADS - \
                                    (currently_locked - currently_failed)

camuthig avatar Jun 29 '20 17:06 camuthig

Have u found any solution for this?? I am also facing issue .

pwnchaurasia avatar May 16 '21 07:05 pwnchaurasia

I do not have a solution for this issue right now @rjsnh1522 . My team moved to using separate processes instead of threads to avoid the issue for now.

camuthig avatar May 26 '21 02:05 camuthig

@rjsnh1522 and @camuthig , I've fixed this problem changing this line: https://github.com/diegobill/django-background-tasks/commit/8fd99c9ad762f635e17f558b601f649893eaa440

now the task is locked just if the lock has not expired

diegobill avatar Sep 07 '22 19:09 diegobill