dd-trace-dotnet icon indicating copy to clipboard operation
dd-trace-dotnet copied to clipboard
verified publisher icon DataDog

Problem with Azure ServiceBus distributed trace

Open brunohdossantos opened this issue 1 year ago • 10 comments

I am unable to use the distributed trace to communicate between services using Azure Service Bus, the trace is only working when a service A communicates with a service B through a web api. I'm using the latest version of dd-trace-dotnet and the Azure ServiceBus SDK. From what I understand after this update https://github.com/DataDog/dd-trace-dotnet/pull/4575/files#diff-426121b5cc1e2b68a3fb5aef68641547dd3488c59332d4a79adf23be9460545d the propagation of the necessary contexts would be done automatically. Has anyone had success in this scenario?

brunohdossantos avatar Mar 20 '24 12:03 brunohdossantos

Hi @brunohdossantos, can you confirm the following information so I can help understand the setup you have:

  1. Are Service A (message sender) and Service B (message receiver) both using the Datadog .NET Tracer v2.38.0 or above, and do they both have environment variable DD_TRACE_OTEL_ENABLED=true? These are the pre-requisites for our Azure Service Bus instrumentation.
  2. Can you also confirm what version of the Azure.Messaging.ServiceBus SDK package you are using?

zacharycmontoya avatar Mar 20 '24 17:03 zacharycmontoya

Hi @zacharycmontoya , yes I have the variable DD_TRACE_OTEL_ENABLED=true, i am using the Azure.Messaging.ServiceBus version - 7.17.14 and Datadog.Trace in version 2.49, the apm version is also 2.49.

When consuming the message, my service performs a _tracer.StartActive, could this be a problem?

brunohdossantos avatar Mar 21 '24 11:03 brunohdossantos

Anyone with the same error? Or with the same scenario and working without problems ?

brunohdossantos avatar Mar 25 '24 16:03 brunohdossantos

Has anyone experienced this problem, had a solution?

brunohdossantos avatar Apr 26 '24 22:04 brunohdossantos

Do we have a solution for this problem?

brunohdossantos avatar Jun 11 '24 02:06 brunohdossantos

I believe this also requires you to also set AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE to true, which injects traceparent into the message properties. This worked for me.

https://docs.datadoghq.com/tracing/trace_collection/compatibility/dotnet-core/#opentelemetry-based-integrations

rnarayana avatar Jun 16 '24 09:06 rnarayana

Here I need to set this environment variable too: AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE

AlvaroRCAbreu avatar Jul 04 '24 16:07 AlvaroRCAbreu

I believe this also requires you to also set AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE to true, which injects traceparent into the message properties. This worked for me.

https://docs.datadoghq.com/tracing/trace_collection/compatibility/dotnet-core/#opentelemetry-based-integrations

Docs don't have this information anymore: image

frankhaugen avatar Aug 19 '24 12:08 frankhaugen

@frankhaugen Apologies, it looks like this callout was moved to here: https://docs.datadoghq.com/opentelemetry/interoperability/instrumentation_libraries/?tab=net#verified-opentelemetry-instrumentation-libraries

AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE will still need to be set to true to work (or via Azure.Experimental.EnableActivitySource)

@brunohdossantos was this by any chance the issue that you were running into?

bouwkast avatar Aug 19 '24 12:08 bouwkast

what's unclear to me is if this is only for a fully OTEL-based setup or if you can flip these two env vars (AZURE_EXPERIMENTAL_ENABLE_ACTIVITY_SOURCE and DD_TRACE_OTEL_ENABLED) and get automatic propagation in a (DD) auto-instrumented app? reading through docs seems to suggest you need to go all-in on an OTEL setup instead

mrtristan avatar May 06 '25 02:05 mrtristan