Celery task changes status when PENDING and SUCCESS/FAILURE but not when STARTED

[axsaucedo]

trafficstars

I have set up django with django_celery_results - everything works, but the task status doesn't get updated when the task begins processing (into STARTED). Is there an extra configuration that needs to be set up for this to be the case?

My current configuration has the following variables in django settings:

# Celery
# ------------------------------------------------------------------------------
if USE_TZ:
    # http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-timezone
    CELERY_TIMEZONE = TIME_ZONE
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-broker_url
CELERY_BROKER_URL = redis://klaw-redis:6379/0
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-result_backend
CELERY_RESULT_BACKEND = django-db
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-accept_content
CELERY_ACCEPT_CONTENT = ["json"]
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-task_serializer
CELERY_TASK_SERIALIZER = "json"
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#std:setting-result_serializer
CELERY_RESULT_SERIALIZER = "json"
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#task-time-limit
# TODO: set to whatever value is adequate in your circumstances
CELERY_TASK_TIME_LIMIT = 30 * 60
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#task-soft-time-limit
# TODO: set to whatever value is adequate in your circumstances
CELERY_TASK_SOFT_TIME_LIMIT = 30 * 60
# http://docs.celeryproject.org/en/latest/userguide/configuration.html#beat-scheduler
CELERY_BEAT_SCHEDULER = "django_celery_beat.schedulers:DatabaseScheduler"

I have also attempted to set CELERY_CACHE_BACKEND=django-cache but also didn't fix it.

My versions include:

django==2.2.9  # pyup: < 3.0  # https://www.djangoproject.com/
celery==4.4.0  # pyup: < 5.0  # https://github.com/celery/celery
django-celery-results==1.2.0 # https://github.com/celery/django-celery-results
django-celery-beat==1.5.0  # https://github.com/celery/django-celery-beat
redis==3.3.11  # https://github.com/andymccurdy/redis-py
django-redis==4.11.0  # https://github.com/niwinz/django-redis

Currently I have to set the task status manually through the django ORM, but this seems to be due to a bug. Is this a common / known bug?

Happy to provide further information.

edit:

Some interesting insight is that the AsycResult of the respective task doesn't update either. Here is an example:

@celery_app.task()
def test_task():
    time.sleep(20)
    return True

Then connecting with python manage.py shell:

> r = test_task.delay()
> r
<AsyncResult: 359f69ea-227d-4289-9d22-41cc611e0f7e>

> r.status
'PENDING'

> r.status
'PENDING'

> r.status
'SUCCESS'

[axsaucedo]

Ok, I've managed to fix this. It only required an extra configuration variable:

# http://docs.celeryproject.org/en/latest/userguide/configuration.html#task-time-limit
CELERY_TASK_TRACK_STARTED = True

It would be great to document this on the django section, as I would assume it's quite a fundamental feature, happy to add a PR if you think it makes sense.

[johnnymetz]

Definitely agree it should be documented somewhere. Tripped me up for a while.

[ZviBaratz]

In my opinion this would fit better in the First Steps with Django page rather than in this package's docs.