Scheduler process sets next_run to UTC but loops over local time

[augustomen]

This may be related to PR https://github.com/Koed00/django-q/pull/470.

My project has USE_TZ = False and Conf.CATCH_UP = False.

Schedules are generally entered in local time. This means that, if a Schedule next_run = today @ 2pm, exactly at 14:01, this query will return a task to run, because 14:00 < timezone.now (https://github.com/Koed00/django-q/blob/master/django_q/cluster.py#L576):

    Schedule.objects.select_for_update()
    .exclude(repeats=0)
    .filter(next_run__lt=timezone.now())

The task is executed as expected. However, when calculating next_run, the following loop will only exit after next_run > UTC time (if I'm UTC-5, that's 7pm).

while True:
    if s.schedule_type == s.MINUTES:
        next_run = next_run.shift(minutes=+(s.minutes or 1))
    ...
    elif s.schedule_type == s.CRON:
        next_run = arrow.get(
            croniter(s.cron, timezone.now()).get_next()
        )
    if Conf.CATCH_UP or next_run > arrow.utcnow():
        break

It's even worse for short CRON schedules, because it gets stuck in the while True loop for ever, since timezone.now() returns local time, and when comparing to arrow.utcnow(), it will always be less. For people on the other side of the world (UTC+n), this will always be True.

You did mention in that PR that "internal times are in UTC", but all times in the project use timezone.now(); I think this loop should not use utcnow().

[MathiasKowoll]

Exactly I modify de code of Django_q each time I update them so it can have the correct time for scheduling.

[rokdd]

I guess I have the same problem.. what is solution to solve this?

[augustomen]

I guess I have the same problem.. what is solution to solve this?

I am NOT proud of this... but I decided to monkey-patch arrow instead:

import django_q
from django_q import cluster
from django.utils import timezone
assert django_q.VERSION <= (1, 3, 4)  # Just for safety
cluster.arrow.utcnow = lambda: cluster.arrow.get(timezone.now())

[rokdd]

import django_q
from django_q import cluster
from django.utils import timezone
assert django_q.VERSION <= (1, 3, 4)  # Just for safety
cluster.arrow.utcnow = lambda: cluster.arrow.get(timezone.now())

Thanks.. where I need to place this?

[augustomen]

Thanks.. where I need to place this?

Anywhere in the initialization of your project (ie, somewhere where it will run when Django starts up). I chose to put it in one of my apps' AppConfig.ready() function, but you can put it top-level in manage.py or urls.py.

[rokdd]

Perfect thanks! Then I was right, now everything runs a bit better, but I also changed some other behaviours in code ;-)