Periodic tasks not working

[lukemaxwell]

I can't seem to get the periodic tasks to execute, or show up in the 'scheduled' queue.

Using the following tasks.py for debug:

#!/usr/bin/env python
# -*- coding: utf-8 -*-
import redis
from tasktiger import TaskTiger
from tasktiger.schedule import periodic

tiger = TaskTiger()


@tiger.task(schedule=periodic(seconds=1), queue='periodic')
def periodic_task():
    """Periodic task."""
    conn = redis.Redis(decode_responses=True)
    conn.incr('period_count', 1)


@tiger.task()
def test_task():
    """Minimal task."""
    print('Hello')

Running tasktiger as shown in docs:

$ PYTHONPATH=. tasktiger

I can run the minimal test_task:

>>> import tasks
>>> tasks.test_task.delay()
<Task <function test_task at 0x7f03aaacfd08>>

I see the output in the log, but there is no evidence of the periodic_task and the redis key 'period_count' never increments:

{"pid": 27295, "queues": [], "exclude_queues": [], "single_worker_queues": [], "event": "ready", "level": "info", "timestamp": "2017-10-26T12:43:35.236665Z"}
{"pid": 27295, "queue": "default", "event": "new queue", "level": "debug", "timestamp": "2017-10-26T12:44:09.818470Z"}
{"pid": 27295, "queue": "default", "src_queue": "queued", "dest_queue": "active", "qty": 1, "event": "moved tasks", "level": "debug", "timestamp": "2017-10-26T12:44:09.819402Z"}
{"pid": 27295, "queue": "default", "child_pid": 27318, "func": "tasks:test_task", "task_id": "c44b198caac44250d06fc5dfc3249960d9eb5bfbe76d8bb848ddb419c8382323", "params": {"args": [], "kwargs": {}}, "event": "processing", "level": "info", "timestamp": "2017-10-26T12:44:09.820179Z"}
Hello
{"pid": 27295, "queue": "default", "attempted": 1, "processed": 1, "event": "processed", "level": "debug", "timestamp": "2017-10-26T12:44:09.823499Z"}
{"pid": 27295, "queue": "default", "task_id": "c44b198caac44250d06fc5dfc3249960d9eb5bfbe76d8bb848ddb419c8382323", "event": "done", "level": "info", "timestamp": "2017-10-26T12:44:09.825171Z"}
{"pid": 27295, "queue": "default", "src_queue": "queued", "dest_queue": "active", "qty": 0, "event": "moved tasks", "level": "debug", "timestamp": "2017-10-26T12:44:09.825592Z"}
{"pid": 27295, "time_total": 60.00032615661621, "time_busy": 0.003787517547607422, "utilization": 0.006312494931645924, "event": "stats", "level": "info", "timestamp": "2017-10-26T12:44:35.237246Z"}

Am I missing something?

By the way, TaskTiger is a great project!

[xuru]

I ran across this issue as well even when passing -m mymodule to tasktiger. I finally figured out a solution that works. I use click for cli commands, and made one to fire off a worker:

@tasks.command("worker")
def start_worker():
    secho(f"Starting worker", fg="green")
    from mymodule.tasks import process_events, process_other_events, process_events_for_specific_email  # register the tasks

    tiger.run_worker(module='mymodule')
    sys.exit(0)

Hope this helps!

[EverCurse]

yeah , i meet same question

[mrname]

I seem to have the same problem as well. Any updates on this? I'm happy to look into it if nobody else is.

[jkemp101]

Does the example in the op work for you if you run tasktiger like this so it processes the periodic queue? PYTHONPATH=. tasktiger -q periodic

[mrname]

Actually @jkemp101 , I think I see what the problem is based on the comments in the test:

https://github.com/closeio/tasktiger/blob/6973a28e7a0c2912a5fb651376a0fd819b9ed713/tests/test_periodic.py#L61

My issue was that I was creating a fresh TaskTiger instance when running the worker instead of using the same one where the periodic task was defined. When I import the same instance that I used when defining the tasks, things work as expected. If somebody else on this issue can confirm that this fixes the issue for them, I will issue a PR for docs updates.

[thomasleveil]

@mrname I was also creating a fresh TaskTiger instance. I confirm this fixes the issue with periodic tasks not being scheduled