[dd-trace 2.11.0] Missing tracking data

[alvarofpp]

Context I have a BFF that uses 'dd-trace' version 2.6.0, this BFF communicates with various services. The problem occurs when I update the version of 'dd-trace' to 2.11.0.

Expected behaviour The tracer should show the request data, including the moments it interacts with other services. Example:

Actual behaviour It is not showing interactions with other services.

Steps to reproduce Take any application that is tracking requests from an endpoint, this application should interact with other services when this endpoint is called, update the version of dd-trace to 2.11.0.

Environment

• Operation system: Platform: GNU/Linux - CPU: Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz - Memory: 16.46G
• Node.js version: v16.15.1
• Tracer version: 2.6.0 and 2.11.0
• Agent version: v7.32.4
• Relevant library versions:
  • express: 4.17.1

[rochdev]

Is the tracer initialized before any instrumented module is imported? This is a requirement for dd-trace to work properly and the behaviour is undefined and can change from version to version of it's not the case. This is usually the issue when spans for a specific integration are completely missing. If they are still reported but in a separate trace, then let me know as that would then be something else completely.

[alvarofpp]

@rochdev The tracer initializes before all other modules. Your location in the code has not changed with the version update.

the behaviour is undefined and can change from version to version of it's not the case

Would you know if you have any attributes in the require('dd-trace').init(); that may have changed this behavior?

[rochdev]

We have re-written the plugin system and in some cases it's more prone to issues when the tracer is not initialized in the right location, at least right now. Can you share your application entry point? I'd like to make sure there aren't any hard to detect issues like hoisting for example.

[mariuszbeltowski]

@rochdev is there any way to enable debug in tracer to see if it connects well with the agent and sends data?

[rochdev]

@mariuszbeltowski Yes but it's extremely verbose, so it's recommended to enable it only to debug issues. This can be configured with DD_TRACE_DEBUG=true.

[rochdev]

@alvarofpp Were you able to solve the original issue? If not, can you try with 2.13.0? We made further changes to the plugin system that could hopefully fix the issue. Otherwise we'll definitely need a reproduction snippet.

[alvarofpp]

@rochdev I'll try 2.13.0 this week or the next, then come back here informing you what happened.

[dyarleniber]

@rochdev I had a similar issue, after upgrading to version 2.11 I noticed that the HTTP route paths were missing from the APM dashboards:

I resolved this by upgrading to version 3.0.0 and moving just the library import to the top of my entry point file. However, I am not initializing the library at the very beginning.

My entry point file looks like this:

import dataDogTracer from 'dd-trace';
import express from 'express';
import winston from 'winston';
// ... some other lib imports

const logger = winston.createLogger({
  // ...
});

dataDogTracer.init({
  logInjection: true,
  runtimeMetrics: true,
  logger
});

// ...

Given your comment above:

"Is the tracer initialized before any instrumented module is imported? This is a requirement for dd-trace to work properly and the behaviour is undefined and can change from version to version of it's not the case. This is usually the issue when spans for a specific integration are completely missing. ..."

It seems to be working fine now, but do you think it's possible that I may have issues again with future versions of the library?

[rochdev]

It seems to be working fine now, but do you think it's possible that I may have issues again with future versions of the library?

@dyarleniber We plan to remove the initialization requirement from 4.0 completely, so for future versions it should be fine. The library will still need to be imported first, but can be initialized anywhere. There are already a few changes in 3.0 to make this work better, so you might just be benefiting from those improvements.

[alvarofpp]

I updated the dd-trace version to 3.0.0 in my project and it worked! I don't know what didn't work between versions >2.6.0 and <3.0.0. Thank you guys, especially @rochdev for the support and @dyarleniber for the report.