celery
Celery Beat 2.8.1 causes DB CPU to max out
After upgrading celery beat from 2.7.0 to 2.8.1, my database seems to start getting spammed with requests from celery beat. I only have a few tasks running each day and I've had celery beat running on this project for over 3 years without any issue so not sure what exactly is causing this. pg_stat_activity did show that this query seemed to be the root of the issue but isn't this just the regular celery beat query that's meant to run regularly. Maybe the interval was increased?
SELECT "django_celery_beat_periodictask"."id", "django_celery_beat_periodictask"."name", "django_celery_beat_periodictask"."task", "django_celery_beat_periodictask"."interval_id", "django_celery_beat_periodictask"."crontab_id", "django_celery_beat_periodictask"."solar_id", "django_celery_beat_periodictask"."clocked_id", "django_celery_beat_periodictask"."args", "django_celery_beat_periodictask"."kwargs", "django_celery_beat_periodictask"."queue", "django_celery_beat_periodictask"."exchange", "django_celery_beat_periodictask"."routing_key", "django_celery_beat_periodictask"."headers", "django_celery_beat_periodictask"."priority", "django_celery_beat_periodictask"."expires", "django_celery_beat_periodictask"."expire_seconds", "django_celery_beat_periodictask"."one_off", "django_celery_beat_periodictask"."start_time", "django_celery_beat_periodictask"."enabled", "django_celery_beat_periodictask"."last_run_at", "django_celery_beat_periodictask"."total_run_count", "django_celery_beat_periodictask"."date_changed",
Not sure if anything was changed recently or if there's any setting I was meant to upgrade post 2.7.0. I've added the insights charts here to show how serious the spike was.
Thanks.
The rise happens after upgrading to 2.8.1 and then falls after downgrading to 2.7.0
I observed something similar, but what was happening for me was that the timezone set for celery was not matching the django project/server timezone. Due to this celery beat has probably created a "poison pill" message in your message broker and some database action is happening over and over and over.
I observed something similar, but what was happening for me was that the timezone set for celery was not matching the django project/server timezone. Due to this celery beat has probably created a "poison pill" message in your message broker and some database action is happening over and over and over.
Both my Django and Celery use UTC as the default timezone. So, I'm not sure if that's gonna be the issue. I'm also not sure how to investigate this.
If there's anyone who can point me to the right log files for dealing with something like this, I'd be grateful.
And obviously it would have to be with something that changed from 2.7 to 2.8.1.