dd-trace-js icon indicating copy to clipboard operation
dd-trace-js copied to clipboard

Indexed spans sometime have x-datadog-sampling-priority of 0 (AUTO_REJECT)

Open DarrenTsung opened this issue 3 years ago • 0 comments

Hi, thanks for the useful library. This is more of a question than a feature request / bug.

We are using tracer.inject to propagate the span context to a sub-process via code like:

      const spanContext = {}
      tracer.inject(span, FORMAT_TEXT_MAP, spanContext)
      env['ORIGIN_SPAN'] = JSON.stringify(spanContext)

And in the child process we read the environment variable and use it to set the parent span.

When we look at the indexed spans, most of them have the child span captured, but some of them do not. After some investigation + looking at the logs, we realized that this is because sometimes the "x-datadog-sampling-priority" is set to 0 in the parent context - where 0 means AUTO_REJECT (i.e. the span was not sampled due to agent rate limiting IIUC). So it seems like the child is doing the right thing because it receives information that the span should not be sampled. But for some reason the outer span is indexed, which seems unexpected.

Do ya'll know what is happening here?

Version: This is happening on v3.3.1 of dd-trace (this behavior was also occurring on a way older version we were running, v0.36.2, but we did an upgrade to check if it was fixed / changed in the latest version of the library).

DarrenTsung avatar Sep 29 '22 20:09 DarrenTsung