APM Traces are not sent

[phortx]

Current behaviour

We're using DataDog for a small grape based API project and have the strange behavior, that everything works well until we set the http.status_code, like this:

# Setup code when app starts
require "datadog/statsd"
require "ddtrace/auto_instrument"

Datadog.configure do |c|
  c.env = ENV["DD_ENV"]
  c.service = "..."

  c.tracing.instrument :grape
  c.tracing.enabled = true
  c.runtime_metrics.enabled = true
end

# ...

# Setup code when a request arrives
@root_span = Datadog::Tracing.active_span
@root_span.set_tag "http.request_id", env["request_id"]

# ...

# Just before request processing is done
@root_span.set_tag Datadog::Tracing::Metadata::Ext::HTTP::TAG_STATUS_CODE, '200'

(or any other value). Then one or two traces appear but then tracing stops and any further traces don't show up any more. In the Diagnostics log nothing appears, so I guess, it's not even sent to the agent.

This just happens for one API endpoint. For other endpoints this works.

When I remove that line, everything works like a charm. We have to set the http.status_code manually because the grape integration doesn't seem to do it.

Things I've tried so far without success:

• Sending a integer instead of a string
• Reducing Puma threads to 0
• Don't setting the tag twice
• Don't send any other http.* tags manually
• Disable some other instrumentations like aws, http and so on
• Calling Datadog::Tracing.keep!
• Activate partial_flush
• Disable Continuous Profiler
• Add validations to ensure nothing invalid is set into that tag
• Setting sampling_rate to 1.0
• Setting runtime_metrics.enabled to false
• Not setting the request_id
• Not requiring datadog/statsd
• ...

Expected behaviour The Trace is sent to DataDog.

Steps to reproduce

Not sure currently. I'm trying to find out, what is causing this. Any help would be nice. I'll update the issue when I have more insights.

Environment

• ddtrace version: 1.13.1
• Configuration block (Datadog.configure ...): see above
• Ruby version: 3.2.2
• Operating system: ECS Fargate (Docker)
• Relevant library versions: grape 1.7.1

[phortx]

It seems like removing that line fixes the issue:

require "ddtrace/auto_instrument"

[phortx]

Feel free to close that issue, when this is not a bug in your opinion :)

[TonyCTHsu]

👋 @phortx , could you provide a sample app to reproduce this?

I would also be interested to understand how require "ddtrace/auto_instrument" could be related.

The code (settings tags) you brought up seems reasonable to me (I don't see any problem at first glance).

Then one or two traces appear but then tracing stops and any further traces don't show up any more.

This just happens for one API endpoint. For other endpoints this works.

These signals seems like this issue is a bit random.

I realized that you mentioned the application is running on ECS Fargate (Docker), but it was not clear to me how you deploy your application. I would suggest to reproduce it locally.