Preventing duplicity while scheduling tasks with celery beat

[alexandernst]

Summary:

I have a task that I run periodically (each minute) via Celery Beat. On occasions, the task will take longer than a minute to finish it's execution, which results in the scheduler adding that task to the queue while the task is already running.

Is there a way I can avoid the scheduler adding tasks to the queue if those tasks are already running?

• Celery Version: Currently using latest, but any version applies
• Celery-Beat Version: Currently using latest, but any version applies

Exact steps to reproduce the issue:

1. Configure a scheduled task to be executed once each minute
2. Make the task sleep for 2 minutes.
3. Check how the queue is growing

Detailed information

N/A

[elchiapp]

There is no builtin way of doing this within beat. I've been using https://github.com/cameronmaske/celery-once and it works pretty well, although it uses Redis expiring keys so it doesn't actually check whether there is already a duplicate task running.

[auvipy]

we might need to consider this to be in the celery core