dd-trace-rb icon indicating copy to clipboard operation
dd-trace-rb copied to clipboard

Propagate current trace context to Active Record async query threads

Open agrobbin opened this issue 11 months ago • 3 comments

What does this PR do?

The Active Record integration (for anything greater than v7.0.0) will now wrap ConnectionPool#build_async_executor with the already-existing ContextCompositeExecutorService from the concurrent-ruby tracer integration, ensuring that the current trace is continued within the thread.

Closes #3465.

Motivation:

When leveraging Active Record's async query functionality (load_async, async_count, etc.), the queries are executed in a different thread, via concurrent-ruby's ThreadPoolExecutor. This causes the span for that query to not show up in the parent trace (often a rack.request).

How to test the change?

I included tests here, but am definitely open to other approaches here!

agrobbin avatar Mar 28 '25 17:03 agrobbin

@marcotc was just wondering if this was something you'd be able to take a look at when you get a chance. It would be great to get this into a release soon! 🤞🏻

agrobbin avatar Sep 10 '25 19:09 agrobbin

@agrobbin, yes! It's pretty much what I've been doing in the last two weeks 😅: https://github.com/DataDog/dd-trace-rb/pull/4872

It required quite a major change in how we flush traces and track background threads (which is a bug that greatly affects the concurrent-ruby instrumentation). More specifically, this didn't work correctly: https://github.com/DataDog/dd-trace-rb/blob/44947b27e86b871e6028d5bece5087a145cabcb5/spec/datadog/tracing/tracer_spec.rb#L993-L1001 when span-1 finished the whole wrapping trace would finish, and span-2 would create a new trace as a fallback that was not associated with the digest. This affects async queries because multiple queries can run in the background for a single load_async, depending on the DB adapter, and those would create new, unrelated traces in the background.

I still need to to fix all failing tests, and add new tests to ensure I didn't break basic span flushing.

marcotc avatar Sep 10 '25 20:09 marcotc

Ohh, wow, way more complicated than I thought! Looking forward to seeing all of that work land. 🎉

agrobbin avatar Sep 10 '25 22:09 agrobbin