Task name is missing from results when task is called in eager mode

[ioreshnikov]

Hey!

I've noticed a weird problem I cannot fix: when I run an explicitly named task in eager mode, the result written in the database doesn't contain task name. The same task being run on a worker does produce the correct result.

I am not sure whether this is related strictly to the current backend, or celery as a whole, though. When jumping through the code I've noticed that the backend takes the task name from the request field:

# django_celery_results/backends/database.py:41
task_name = getattr(request, 'task', None)

However, the request as it is constructed by Task.apply() does not contain task name as a field, so the corresponding value is None.

In any case, there's this disparity of expectations between Task class and the current backend and I don't quite understand how to make it work. Is there a known work-around?

Best, Ivan

[CDavantzis]

It's due to this logic: https://github.com/celery/django-celery-results/blob/5f507c8606736e54b0dcddc6fa03642721217423/django_celery_results/backends/database.py#L72

I found a work-around by comparing the task.request contexts between eager and non-eager tasks. You can optionally subclass celery.Task and do the patch in before_start or some other method that makes sense for your use-case.

import celery
import celery.states


def maybe_patch_eager_task(task: celery.Task):
    if not task.request.is_eager:
        return

    # patch task name so it appears in django-celery-results.
    if not hasattr(task.request, "task"):
        setattr(task.request, "task", task.name)

    # optional step to add started state to eager tasks.
    if (
        task.app.conf.task_store_eager_result
        and not task.ignore_result
        and not task.request.ignore_result
        and task.track_started
    ):
        task.update_state(state=celery.states.STARTED)


@celery.shared_task(bind=True, track_started=True)
def your_task(task: celery.Task, *args, **kwargs):
    maybe_patch_eager_task(task)
    ...

[auvipy]

I am open to any possible solution anyone propose

[dalmasonto]

Hello, was having the same issue but the solution I got only solved one not the other. I was dynamically loading/settings tasks using django celery beat and database scheduler but after the task was run, the task_name was missing and the periodic_task_name I got a workaround that saved my day abit that is adding the following line in my settings file

# Celery Results settings
CELERY_RESULT_EXTENDED = True

The line helped me get to where I was heading but still could load the periodic_task_name all the other properties were loading successfully afterwards except for that one.

Any help for loading periodic_task_name can help me much

[dalmasonto]

Hello guys, am back with a different finding after a number of hours of trying to get periodic task name to show up, one might ask why I wanted this, in my use case, I wanted to filter down the dynamic tasks from django celery beat per user. I came to realize that, besides using the CELERY_RESULTS_EXTENDED = True to make sure that all the rest of the metadata gets sent together with the task when it is being created, there was one extra external factor required to make this one work well, ie periodic_task_name to be sent over and that was using redis instead of rabbitMQ

I had to install redis and this was the result Where periodic task name is - I have used rabbitMQ where there is a periodic task name, I have used redis

In your requirements file you will need to have

Django==4.0.4
django-celery-beat==2.5.0
django-celery-results==2.5.1
redis==4.5.5

Django has to be that version 4.0.4 Otherwise celery beat won't make migrations. The rest can be of different versions though am not so sure.

[clayheaton]

I'm using redis and just set up Celery with Django. I have a single test task:

@shared_task
def add(x, y):
    return x + y

The task executes properly and the result appears in the result backend. However, task_name is always null in the database.

Here are my Celery settings:

CELERY_RESULT_BACKEND = "django-db"
CELERY_BROKER_URL = "redis://redis:6379/2"
CELERY_TIMEZONE = "UTC"
CELERY_TASK_TRACK_STARTED = True
CELERY_TASK_TIME_LIMIT = (60 * 60)
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True
CELERY_BEAT_SCHEDULER = "django_celery_beat.schedulers:DatabaseScheduler"
CELERY_RESULTS_EXTENDED = True

I also tried the following and it did not record the name.

@shared_task(name="myapp.tasks.add")
def add(x, y):
    return x + y

I'm assuming this is part of the same bug? Any ideas for a workaround to get the task name in the results backend?

[vi-klaas]

I have CELERY_RESULTS_EXTENDED = True and celery eager False*and I don't get the periodic_task _name when using RabbitMQ as well. I am thinking of a workourand with using @shared_task(bind=True), and then try to update the periodic_task_name inside the task (or maybe creating a custom decorator for that).

[vi-klaas]

I also observe that self.update_state(state='PROGRESS', meta=...) updates the Meta, but the state is displayed as - Could it be related?