django-celery-beat
django-celery-beat copied to clipboard
Django Celery Beat not sending tasks to Celery Worker
Summary:
Celery-beat not sending the tasks to celery worker for execution.
- Celery Version: 5.2.3
- Celery-Beat Version: 2.2.1
Exact steps to reproduce the issue:
- Start the Django project using docker-compose up -d
- Stop the Django project using docker-compose down
- Restart the Django project using docker-compose up -d
Detailed information
When I first start the Docker, all the tasks run without any issue. If I stop the docker (i.e. docker-compose down) and then restart the docker (i.e. docker-compose up), celery-beat does not send the tasks to the celery worker in order for them to get executed.
If I visit the Admin panel and disable the tasks and then re-enable them it starts working!
Also if I do not use the django_celery_beat.schedulers:DatabaseScheduler and allow celery beat to use the default scheduler it also works. Even though this method works, is not the best scenario since the tasks are not longer editable in Django admin panel
Is there a solution to this issue?
Postgresql is not part of the docker-compose. I have a native PostgreSQL 10.5 installation.
settings.py
CACHES = {
"default": {
"BACKEND": "django_redis.cache.RedisCache",
"LOCATION": "redis://redis:6379",
"OPTIONS": {
"CLIENT_CLASS": "django_redis.client.DefaultClient",
"CONNECTION_POOL_KWARGS":{"max_connections":50, "retry_on_timeout": True}
}
}
}
SESSION_ENGINE = "django.contrib.sessions.backends.cache"
SESSION_CACHE_ALIAS = "default"
CELERY_TIMEZONE = "Europe/Amsterdam"
CELERY_TASK_TRACK_STARTED = True
CELERY_TASK_TIME_LIMIT = 30 * 60
CELERY_BROKER_URL="redis://redis:6379"
CELERY_CACHE_BACKEND = 'default'
CELERY_BEAT_SCHEDULE = {
"sendemails_task":{
"task":"demoapp.tasks.testing",
"schedule":crontab(minute='*/2')
},
}
docker-compose.yml
services:
redis:
image: redis
restart: unless-stopped
expose:
- 6379
web:
build:
context: ./app
dockerfile: Dockerfile.prod
restart: unless-stopped
command: gunicorn Project.wsgi:application --bind 0.0.0.0:8001 --workers=4 --preload
volumes:
- static_volume:/home/app/web/staticfiles
- ./uploads/:/home/app/web/uploads/
expose:
- 8001
env_file:
- ./.env.prod
celery:
build:
context: ./app
dockerfile: Dockerfile.prod
command: celery -A Project worker -l info
volumes:
- ./app/:/usr/src/app/
env_file:
- ./.env.prod
depends_on:
- web
- redis
celery-beat:
build:
context: ./app
dockerfile: Dockerfile.prod
command: celery -A Project beat -l debug --scheduler django_celery_beat.schedulers:DatabaseScheduler
volumes:
- ./app/:/usr/src/app/
env_file:
- ./.env.prod
depends_on:
- web
- redis
- celery
volumes:
static_volume:
tasks.py
from __future__ import absolute_import, unicode_literals
from celery.utils.log import get_task_logger
from celery import shared_task
from django.core.cache import cache
import time
import traceback
@shared_task
def testing():
print('test')
celery.py
from __future__ import absolute_import, unicode_literals
import os
from celery import Celery
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'Project.settings')
app = Celery('Project')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.autodiscover_tasks()
celery beat log
[2022-01-31 16:24:56,138: DEBUG/MainProcess] beat: Ticking with max interval->5.00 seconds
[2022-01-31 16:24:56,217: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:01,223: DEBUG/MainProcess] beat: Synchronizing schedule...
[2022-01-31 16:25:01,223: DEBUG/MainProcess] Writing entries...
[2022-01-31 16:25:01,241: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:06,260: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:11,282: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:16,303: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:21,433: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:26,451: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:31,472: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:36,491: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:41,511: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:46,533: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:51,615: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:25:56,636: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:01,654: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:06,669: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:11,692: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:16,710: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:21,733: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:26,754: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:31,770: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:36,949: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:41,971: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:46,989: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:52,010: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:26:57,030: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:02,051: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:07,073: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:12,093: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:17,115: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:22,137: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:27,155: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:32,176: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:37,208: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:42,228: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:47,245: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:52,266: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:27:57,288: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:02,293: DEBUG/MainProcess] beat: Synchronizing schedule...
[2022-01-31 16:28:02,294: DEBUG/MainProcess] Writing entries...
[2022-01-31 16:28:02,395: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:07,413: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:12,431: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:17,478: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:22,499: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:27,522: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:32,545: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:37,564: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:42,585: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:47,602: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:52,614: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:28:57,629: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:02,653: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:07,991: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:13,011: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:18,029: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:23,097: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:28,115: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:33,135: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:38,153: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:43,167: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:48,184: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:53,202: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:29:58,222: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:03,238: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:08,258: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:13,270: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:18,288: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:23,305: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:28,360: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:33,382: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:38,403: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:43,424: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:48,440: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:53,459: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:30:58,490: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:03,496: DEBUG/MainProcess] beat: Synchronizing schedule...
[2022-01-31 16:31:03,496: DEBUG/MainProcess] Writing entries...
[2022-01-31 16:31:03,510: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:08,572: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:13,592: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:18,616: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:23,637: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:28,660: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:33,682: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:38,719: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:43,822: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:48,840: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:53,857: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:31:58,877: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:03,898: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:08,919: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:13,940: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:18,959: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:24,064: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:29,087: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:34,106: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:39,127: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:44,145: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:49,166: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:54,184: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:32:59,207: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:04,344: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:09,366: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:14,383: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:19,400: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:24,418: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:29,437: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:34,459: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:39,477: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:44,499: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:49,522: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:54,542: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:33:59,563: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:04,568: DEBUG/MainProcess] beat: Synchronizing schedule...
[2022-01-31 16:34:04,568: DEBUG/MainProcess] Writing entries...
[2022-01-31 16:34:04,580: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:09,598: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:14,621: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:19,642: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:24,662: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:29,683: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:34,702: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:39,722: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:44,741: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:49,759: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:54,778: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:34:59,800: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:04,819: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:09,838: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:14,860: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:19,982: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:25,001: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:30,018: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:35,039: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:40,058: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:45,078: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:50,098: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:35:55,118: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:00,140: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:05,159: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:10,181: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:15,195: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:20,213: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:25,233: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:30,338: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:35,360: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:40,382: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:45,403: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:50,425: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:36:55,487: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:00,507: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:05,511: DEBUG/MainProcess] beat: Synchronizing schedule...
[2022-01-31 16:37:05,511: DEBUG/MainProcess] Writing entries...
[2022-01-31 16:37:05,523: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:10,541: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:15,568: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:20,585: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:25,607: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:30,694: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:35,713: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:40,734: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:45,796: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:50,814: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:37:55,834: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:00,854: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:05,874: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:10,895: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:15,913: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:20,935: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:25,955: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:30,975: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:35,996: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:41,014: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:46,033: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:51,052: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:38:56,068: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:01,087: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:06,101: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:11,122: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:16,176: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:21,193: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:26,214: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:31,235: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:36,251: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:41,266: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:46,282: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:51,299: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:39:56,319: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:01,338: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:06,343: DEBUG/MainProcess] beat: Synchronizing schedule...
[2022-01-31 16:40:06,343: DEBUG/MainProcess] Writing entries...
[2022-01-31 16:40:06,357: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:11,377: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:16,399: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:21,421: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:26,438: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:31,459: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:36,481: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:41,498: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
[2022-01-31 16:40:46,514: DEBUG/MainProcess] beat: Waking up in 5.00 seconds.
Is there any news about this bug?
@ilourt @pisaia01 We are seeing the same problem. With docker. First run works. On subsequent runs unless we disable/enable it does not works. We tried changing other parameters of the job but nothing seems to solve this.
We can also see this in development environment without docker.
In development we left the worker, admin and flower running and only restarted celery beat
. Same behavior. We need to disable/enable to get this working.
How can we get a consistent behavior with celery-beat?
Don't use this django-celery-beat .default PersistentScheduler Works fine, django-celery-bea package will just make you pull your hairs.
Can confirm it still happens for v2.5.0
. Removing django_celery_beat.schedulers:DatabaseScheduler
works, but will un-link django_celery_results
's periodic task name (will now be None).
Hi @ all, i'm facing the same issue with my setup which is similar to the initial reported one. Is there any progress on this topic or workaround for this (except not using the DatabaseScheduler)? Thanks & BR
Meanwhile i have tested this without using docker. celery 5.2.3 django-celery-beat 2.2.1
Get the same result - Task was not be send to celery worker.
$ celery -A project beat --scheduler django_celery_beat.schedulers:DatabaseScheduler -l DEBUG
celery beat v5.2.3 (dawn-chorus) is starting.
__ - ... __ - _
LocalTime -> 2023-08-09 12:33:55
Configuration ->
. broker -> amqp://rabbituser:**@localhost:5672//
. loader -> celery.loaders.app.AppLoader
. scheduler -> django_celery_beat.schedulers.DatabaseScheduler
. logfile -> [stderr]@%DEBUG
. maxinterval -> 5.00 seconds (5s)
[2023-08-09 12:33:55,476: DEBUG/MainProcess] DatabaseScheduler: initial read
[2023-08-09 12:33:55,476: DEBUG/MainProcess] Writing entries...
[2023-08-09 12:33:55,477: DEBUG/MainProcess] DatabaseScheduler: Fetching database schedule
[2023-08-09 12:33:55,484: DEBUG/MainProcess] Current schedule:
<ModelEntry: celery.backend_cleanup celery.backend_cleanup(*[], **{}) <crontab: 0 4
* *
* (m/h/d/dM/MY), Europe/Vienna>
>
<ModelEntry: Test BEAT in DEV project.apps.administration.tasks.test_celery_beat_in_dev(*[], **{}) <freq: 10.00 seconds>>
[2023-08-09 12:34:00,510: DEBUG/MainProcess] Writing entries...
$ celery -A project worker --scheduler django_celery_beat.schedulers:DatabaseScheduler -l DEBUG
-------------- celery@hostxyz v5.2.3 (dawn-chorus)
--- ***** -----
-- ******* ---- Linux-5.15.0-76-generic-x86_64-with-glibc2.31 2023-08-09 12:37:24
- *** --- * ---
- ** ---------- [config]
- ** ---------- .> app: project:0x7fd74228d700
- ** ---------- .> transport: amqp://rabbituser:**@localhost:5672//
- ** ---------- .> results:
- *** --- * --- .> concurrency: 12 (prefork)
-- ******* ---- .> task events: OFF (enable -E to monitor tasks in this worker)
--- ***** -----
-------------- [queues]
.> celery exchange=celery(direct) key=celery
[tasks]
. celery.accumulate
. celery.backend_cleanup
. celery.chain
. celery.chord
. celery.chord_unlock
. celery.chunks
. celery.group
. celery.map
. celery.starmap
[2023-08-09 12:37:25,431: INFO/MainProcess] Connected to amqp://rabbituser:**@127.0.0.1:5672//
[2023-08-09 12:37:25,437: INFO/MainProcess] mingle: searching for neighbors
[2023-08-09 12:37:26,456: INFO/MainProcess] mingle: all alone
[2023-08-09 12:37:26,484: WARNING/MainProcess] /home/userxyz/.pyenv/versions/3.9.6/envs/venv_3.9.6/lib/python3.9/site-packages/celery/fixups/django.py:203: UserWarning: Using settings.DEBUG leads to a memory
leak, never use this setting in production environments!
warnings.warn('''Using settings.DEBUG leads to a memory
I have created a Interval schedule which starts every 10 seconds to print out some text. Only when i disable and re enable the task, it proceeds.
This still happens with
- celery 5.3.1
- django-celery-beat 2.5.0
Manual or automatic runs were not happening. What worked for us was changing an interval to a crontab and running it again.
I don't feel confident at all with this though, and would appreciate progress with this issue.
same here, it's not reliable at all, I keep enabling/disabling until it works any progress ?
me too
таже хуйня
Oh, everything is good. just switched from sqlite db to postgre db and fixed it. 😅
Oh, everything is good. just switched from sqlite db to postgre db and fixed it. 😅
Not my case, I've been using Postgres from the start 😞
Is there any recent progress on resolving this issue? Alternatively, does anyone know of a workaround? I'm encountering the same problem and need to enable/disable it to make it work again.
I was having the same issue where the tasks weren't being executed by the beat, and the problem has been fixed after changing this:
@app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
sender.add_periodic_task(crontab(), test())
by this:
app.conf.beat_schedule = {
'taks-id': {
'task': 'full.task.path.test',
'schedule': crontab(),
},
}
edit:
the celery app is being imported from the app configured on main.celeryapp
with the proper configuration:
app = Celery("main")
app.config_from_object("main.settings.celery", namespace="CELERY")
app.autodiscover_tasks()