celery
Using django-celery-beat， Scheduled tasks are triggered repeatedly
celery==4.4.6
django-celery-beat==2.1.0
Django==3.1.2
django-celery-results==2.0.0
Set a timing policy: execute the task at 00:10 every day task_A look at the beat log shows that
[2021-02-26 00:10:00,052: INFO/MainProcess] Scheduler: Sending due task task_A
[2021-02-26 00:10:05,194: INFO/MainProcess] Scheduler: Sending due task task_A
celery log:
[2021-02-26 00:10:00,061: INFO/MainProcess] Received task: task_A[e30e7941-a7da-4a9f-90d9-b125f8e4c46] expires:[2021-03-29 16:01:59.944841+00:00]
[2021-02-26 00:10:00,120: INFO/ForkPoolWorker-5] Task task_A[e30e7941-a7da-4a9f-90d9-fb125f8e4c46] succeeded in 0.05772967217490077s: True
[2021-02-26 00:10:05,233: INFO/MainProcess] Received task: task_A[c9e7ce40-664d-4bf8-a14d-84651a5885eb] expires:[2021-03-29 16:02:05.186303+00:00]
[2021-02-26 00:10:05,298: INFO/ForkPoolWorker-5] Task task_A[c9e7ce40-664d-4bf8-a14d-84651a5885eb] succeeded in 0.06434300076216459s: True
other configuration parameters:
BROKER_BACKEND = 'redis'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TASK_SERIALIZER = 'json'
CELERY_TASK_RESULT_EXPIRES = 60 * 60 * 24
CELERYD_CONCURRENCY = 4
CELERYD_MAX_TASKS_PER_CHILD = 4
CELERYD_PREFETCH_MULTIPLIER = 4
CELERYD_FORCE_EXECV = True
# BROKER_TRANSPORT_OPTIONS = {'visibility_timeout': 90}
CELERY_CREATE_MISSING_QUEUES = True
CELERY_IGNORE_RESULT = True
CELERY_SEND_EVENTS = False
CELERY_DISABLE_RATE_LIMITS = False
# CELERY_EVENT_QUEUE_EXPIRES = 60
CELERY_TIMEZONE = TIME_ZONE
CELERY_ENABLE_UTC = False
CELERY_RESULT_BACKEND = 'django-db'
CELERY_BEAT_SCHEDULER = 'django_celery_beat.schedulers:DatabaseScheduler'
can you try https://github.com/celery/django-celery-beat/releases/tag/v2.2.0 & https://github.com/celery/django-celery-beat/commit/bab79d9d7ed76884fcb6ae5b476fb526f9b5fb06
can you try https://github.com/celery/django-celery-beat/releases/tag/v2.2.0 & bab79d9
Current version：
django-celery-beat==2.2.0
Still having problems with tasks triggering repeatedly= =
[2021-07-27 00:05:00,019: INFO/MainProcess] Scheduler: Sending due task generate_ss (rep.tasks.ss) [2021-07-27 00:05:20,217: INFO/MainProcess] Scheduler: Sending due task generate_ss (rep.tasks.ss) [2021-07-27 00:05:20,261: INFO/MainProcess] Scheduler: Sending due task generate_ss (rep.tasks.ss)
beat_loop_interval=20
The task at 00:05:00 has been executed successfully and the execution time is <20s. At 00:05:20, the task has been generated twice more. The two different task_id
There is also a different kind of problem from the above
[2021-07-27 11:32:21,489: INFO/MainProcess] DatabaseScheduler: Schedule changed. [2021-07-27 11:32:41,576: INFO/MainProcess] DatabaseScheduler: Schedule changed. [2021-07-27 11:33:01,661: INFO/MainProcess] DatabaseScheduler: Schedule changed. [2021-07-27 11:33:01,700: INFO/MainProcess] DatabaseScheduler: Schedule changed. [2021-07-27 11:33:01,738: INFO/MainProcess] DatabaseScheduler: Schedule changed. [2021-07-27 11:33:21,795: INFO/MainProcess] DatabaseScheduler: Schedule changed. [2021-07-27 11:33:21,833: INFO/MainProcess] DatabaseScheduler: Schedule changed. [2021-07-27 11:33:21,872: INFO/MainProcess] DatabaseScheduler: Schedule changed.
interval:20s Irregularly triggered three times at the same time
I also have this similar problem: beat sending task constantly, but only built-in task. This is issue link in django-celery-result project: https://github.com/celery/django-celery-results/issues/275
These are logs:
[2022-01-07 01:51:27,676: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup) [2022-01-07 01:51:27,684: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup) [2022-01-07 01:51:27,689: INFO/ForkPoolWorker-16801] Task celery.backend_cleanup[571c20ea-012f-4cce-8778-cf83c591c6b9] succeeded in 0.022669575992040336s: None [2022-01-07 01:51:27,690: INFO/ForkPoolWorker-16800] Task celery.backend_cleanup[73f742c1-796e-4377-9619-4fcf0d3e7e1a] succeeded in 0.012219233001815155s: None [2022-01-07 01:51:27,691: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup) [2022-01-07 01:51:27,693: INFO/MainProcess] Task celery.backend_cleanup[5370d550-161e-4d80-9309-d8e84b18fca4] received [2022-01-07 01:51:27,696: INFO/MainProcess] Task celery.backend_cleanup[63c4e1f4-141e-4371-bff1-5e86533258d3] received [2022-01-07 01:51:27,698: INFO/MainProcess] Task celery.backend_cleanup[5d89a5e4-faba-4814-89f0-939f8be2c4bc] received [2022-01-07 01:51:27,700: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup) [2022-01-07 01:51:27,706: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup) [2022-01-07 01:51:27,708: INFO/ForkPoolWorker-16801] Task celery.backend_cleanup[4e27da29-82af-499a-83ea-13414258494b] succeeded in 0.016296524001518264s: None [2022-01-07 01:51:27,708: INFO/ForkPoolWorker-16800] Task celery.backend_cleanup[08bc4384-6efb-4e51-ae0e-926444cc7195] succeeded in 0.017958137003006414s: None [2022-01-07 01:51:27,713: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup) [2022-01-07 01:51:27,720: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup) [2022-01-07 01:51:27,726: INFO/ForkPoolWorker-16801] Task celery.backend_cleanup[8e16bf97-e377-4cdb-b52d-d3de11aac86c] succeeded in 0.015769034012919292s: None [2022-01-07 01:51:27,728: INFO/Beat] Scheduler: Sending due task celery.backend_cleanup (celery.backend_cleanup) [2022-01-07 01:51:27,731: INFO/ForkPoolWorker-16800] Task celery.backend_cleanup[4f656b98-0fe7-4cfd-a4e8-418ce05ecc51] succeeded in 0.01994775200728327s: None [2022-01-07 01:51:27,733: INFO/MainProcess] Task celery.backend_cleanup[f3edf723-f43e-4e3e-81ce-c1ff6093801a] received [2022-01-07 01:51:27,736: INFO/MainProcess] Task celery.backend_cleanup[31745008-ad0a-46fa-ba12-2bd00ad95bf1] received [2022-01-07 01:51:27,738: INFO/MainProcess] Task celery.backend_cleanup[59cfaeff-58bf-4d85-ad08-863d4f96f912] received [2022-01-07 01:51:27,739: INFO/MainProcess] Task celery.backend_cleanup[d0fc46d8-10c1-4c40-a644-e0561b667a49] received
I also have this similar problem: beat sending task constantly, but only built-in task. This is issue link in django-celery-result project: celery/django-celery-results#275
what about https://github.com/celery/django-celery-results/issues/275#issuecomment-1010580760 ?
Django settings USE_TZ = True and set Celery config enable_utc = True, the problem goes away. It looks like a problem about time zone and utc settings.
