celerybeatredis
celerybeatredis copied to clipboard
kongluoxing
celerybeat redis ignores CELERYBEAT_SCHEDULE setting
$ celery -A apps.myproject beat -l info
celery beat v3.1.17 (Cipater) is starting.
Configuration ->
. broker -> redis://localhost:6379/0
. loader -> celery.loaders.app.AppLoader
. scheduler -> celery.beat.PersistentScheduler
. db -> celerybeat-schedule
. logfile -> [stderr]@%INFO
. maxinterval -> now (0s)
[2014-12-23 20:30:47,539: INFO/MainProcess] beat: Starting...
[2014-12-23 20:30:57,690: INFO/MainProcess] Scheduler: Sending due task update-tickets-index (helpdesk.tasks.update_tickets_index)
[2014-12-23 20:31:07,713: INFO/MainProcess] Scheduler: Sending due task update-tickets-index (helpdesk.tasks.update_tickets_index)
[2014-12-23 20:31:17,714: INFO/MainProcess] Scheduler: Sending due task update-tickets-index (helpdesk.tasks.update_tickets_index)
[2014-12-23 20:31:27,717: INFO/MainProcess] Scheduler: Sending due task update-tickets-index (helpdesk.tasks.update_tickets_index)
$ celery -A apps.myproject beat -l info -S celerybeatredis.schedulers.RedisScheduler
celery beat v3.1.17 (Cipater) is starting.
Configuration ->
. broker -> redis://localhost:6379/0
. loader -> celery.loaders.app.AppLoader
. scheduler -> celerybeatredis.schedulers.RedisScheduler
. logfile -> [stderr]@%INFO
. maxinterval -> now (0s)
[2014-12-23 20:31:50,722: INFO/MainProcess] beat: Starting...
[2014-12-23 20:31:50,723: INFO/MainProcess] backend scheduler using redis://localhost:6379/0
my configuration:
BROKER_URL = 'redis://localhost:6379/0'
BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 3600} # 1 hour.
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'
CELERY_TASK_SERIALIZER = 'json'
CELERY_ACCEPT_CONTENT = ['json',]
CELERY_REDIS_SCHEDULER_URL = 'redis://localhost:6379/0'
CELERY_TIMEZONE = TIME_ZONE
CELERYBEAT_SCHEDULE = {
'update-tickets-index': {
'task': 'helpdesk.tasks.update_tickets_index',
'schedule': timedelta(seconds=10),
'args': ()
},
}
CELERY_REDIS_SCHEDULER_KEY_PREFIX = 'tasks:project:development:meta:'
related packages:
django-redis==3.7.2
celery==3.1.17
django-celery-transactions==0.2.0
celerybeat-redis==0.0.7
here's the debug output:
$ celery -A apps.myproject beat -l debug -S celerybeatredis.schedulers.RedisScheduler
celery beat v3.1.17 (Cipater) is starting.
__ - ... __ - _
Configuration ->
. broker -> redis://localhost:6379/0
. loader -> celery.loaders.app.AppLoader
. scheduler -> celerybeatredis.schedulers.RedisScheduler
. logfile -> [stderr]@%DEBUG
. maxinterval -> now (0s)
[2014-12-23 20:39:16,164: DEBUG/MainProcess] Setting default socket timeout to 30
[2014-12-23 20:39:16,164: INFO/MainProcess] beat: Starting...
[2014-12-23 20:39:16,164: INFO/MainProcess] backend scheduler using redis://localhost:6379/0
[2014-12-23 20:39:16,164: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes
[2014-12-23 20:39:16,165: DEBUG/MainProcess] beat: Waking up in 5.00 minutes.
why does it wake in 5 min.? is it some redis limitation?
yep, it ignore CELERYBEAT_SCHEDULE，because you must make schedule in redis。CELERYBEAT_SCHEDULE only reads by the builtin scheduler. please read README, it explained how to set your scheduler.
Hi, I just encountered the same problem, by following the project readme.
I tried to start celery beat with celerybeatredis.schedulers.RedisScheduler but it didn't load the scheduled tasks in celeryconfig.py, but when I start the beat using build-in scheduler, it works.
This section says that celerybeat-redis will load CELERYBEAT_SCHEDULE into redis, should we delete it? @kongluoxing
starting celery beat using built-in scheduler
$ celery -A tackle.app:tackleapp beat --loglevel=DEBUG
celery beat v3.1.23 (Cipater) is starting.
__ - ... __ - _
Configuration ->
. broker -> redis://someredis.net:6379/0
. loader -> celery.loaders.app.AppLoader
. scheduler -> celery.beat.PersistentScheduler
. db -> celerybeat-schedule
. logfile -> [stderr]@%DEBUG
. maxinterval -> now (0s)
[2016-04-21 14:37:26,917: DEBUG/MainProcess] Setting default socket timeout to 30
[2016-04-21 14:37:26,918: INFO/MainProcess] beat: Starting...
[2016-04-21 14:37:26,935: DEBUG/MainProcess] Current schedule:
<Entry: monitor-eru-apps tackle.tasks.trigger_monitor_eru_apps() <crontab: * * * * * (m/h/d/dM/MY)>
<Entry: celery.backend_cleanup celery.backend_cleanup() <crontab: 0 4 * * * (m/h/d/dM/MY)>
[2016-04-21 14:37:26,936: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes
[2016-04-21 14:37:26,973: DEBUG/MainProcess] beat: Waking up in 33.02 seconds.
starting celery beat using RedisScheduler
$ celery -A tackle.app:tackleapp beat -S celerybeatredis.schedulers.RedisScheduler --loglevel=DEBUG
celery beat v3.1.23 (Cipater) is starting.
__ - ... __ - _
Configuration ->
. broker -> redis://someredis:6379/0
. loader -> celery.loaders.app.AppLoader
. scheduler -> celerybeatredis.schedulers.RedisScheduler
. db -> redis://someredis.net
. logfile -> [stderr]@%DEBUG
. maxinterval -> now (0s)
[2016-04-21 14:37:35,055: DEBUG/MainProcess] Setting default socket timeout to 30
[2016-04-21 14:37:35,055: INFO/MainProcess] beat: Starting...
[2016-04-21 14:37:35,055: INFO/MainProcess] backend scheduler using redis://someredis.net
[2016-04-21 14:37:35,062: DEBUG/MainProcess] beat: Ticking with max interval->5.00 minutes
[2016-04-21 14:37:35,062: DEBUG/MainProcess] RedisScheduler: Fetching database schedule
[2016-04-21 14:37:35,064: DEBUG/MainProcess] DB schedule : {}
[2016-04-21 14:37:35,064: DEBUG/MainProcess] beat: Waking up in 5.00 minutes.
^C[2016-04-21 14:37:35,752: INFO/MainProcess] Writing modified entries...
[2016-04-21 14:37:35,752: INFO/MainProcess] Writing modified entries...
