Standalone Celery and traces

[francoisgfx]

Hello,

We have 10 celery services (apps) working together with the same rabbitmq. Some task are send from outside the services and some tasks are called between them. To track it we are using the apm standalone celery approach which is recommended. But it seems that the distributed trace implemented in this plugin is not suited for this. I'm wondering if its because celery is treated like a module rather than a framework ?

The results is that we can see the list of all our services. but some are empty, with no transaction at all. And when I'm looking at some services they have their own transaction listed as well as the one from other services. I'm guessing its the way it as been design for Django but not to work in a meshing of services ?

What I would expect is seeing every transaction listed in its own celery services. And when looking at a specific transaction, if a trace has been pass to another service, seeing it also in the waterfall of that transaction. Right now it is quite a mess, but I'm not sure why.

Environment (please complete the following information)

• OS: Linux
• Python version: 2.7
• Framework and version : Celery 4.4.7
• Agent version: elasticapm 5.10.1

Additional context

ServiceA and ServiceB are listed in Kibana APM. Both services are Celery apps.

import elasticapm
from elasticapm import Client, instrument, events
from elasticapm.contrib.celery import register_exception_tracking, register_instrumentation

instrument()
apm_client = Client(service_name=app_name)
register_instrumentation(apm_client)
register_exception_tracking(apm_client)
app = Celery()

if ServiceB is called from ServiceA, transactions in serviceA will be empty, everything will be listed in ServiceB. So in ServiceB transactions list I'll see ServiceB.method and ServiceA.method

[beniwohli]

Hi @francoisgfx! We recently merged #1371, which should enable distributed tracing between different celery services (the title of the PR suggests that it is Django-specific, but that's not the case).

We'll release version 6.8 of the agent later this week.