opentelemetry.io icon indicating copy to clipboard operation
opentelemetry.io copied to clipboard

Need of docs/demo showing end-to-end tracing in distributed services

Open psk001 opened this issue 2 years ago • 7 comments

What needs to be changed? For inter-services context-propagation, in nodejs backend, there is a dearth of available docs or relevant examples.Docs available on W3C docs , or blogs of some companies like DoorDash are quite hard to understand for new joiners.

Additional context An end to end demo or document showing how to implement distributed tracing in services using nodejs in backend, with Trace-Context HTTP Propagator is much needed

psk001 avatar Oct 13 '22 12:10 psk001

I would say this isn't really a docs thing, at least not yet. This is what the community demo is intended to solve: https://opentelemetry.io/community/demo/

It is a work in progress but intended to launch later this month. At some point docs will likely have to establish best practices for end users, including "how to do tracing end-to-end", but those are extremely complex topics that are often better described by books or demo apps.

cartermp avatar Oct 13 '22 21:10 cartermp

Thanks for quick response @cartermp . What i'm looking for is any docs or example for context propagation in nodejs distributed service. As linked above, the W3C docs is quite complex and it does not have relevant examples. It does have header examples for the requesting service but nothing as such for responding one. Further I do have traceparent header in the request but i'm not finding any way to extract trace context and create child spans next.
If you could reference any relevant docs/blogs/github repos, that would be of great help. Cheers

psk001 avatar Oct 14 '22 07:10 psk001

So I'm not entirely clear what your goal is. Are you using instrumentation libraries? That's how context propagates automatically from service to service.

cartermp avatar Oct 14 '22 16:10 cartermp

image

In service A, i call service B as-

image

I pass the traceparent header as specified in W3C docs. Now in Service b , i want to use traceparent header to create new child span, so as to get a complete flow of request - response in jaeger. I did not find any example or method in opentelemetry to extract trace context from the trace id and span id from the traceparent header. I found this repo but even this is nor about inter-services custom trace propgation. How should I pull this off? Thanks

psk001 avatar Oct 14 '22 20:10 psk001

So unless you have a requirement that you can't use instrumentation packages for Node, I highly, highly recommend not trying to manually propagate context across services. The libraries handle this for you. The auto-instrumentations-node pulls in everything available and should be able to do what you need there.

Now, if you must manually propagate context because you cannot add additional libraries, then this is probably just a duplicate of #1315. Eventually we'll likely have docs for how to manually do context propagation, but the vast majority of people just do this with libraries at their service boundaries, so it hasn't been a pressing need.

cartermp avatar Oct 14 '22 21:10 cartermp

👍 what @cartermp said, and two addendums:

  • and if the library you are using is not having otel natively or their is no instrumentation library for it, instead of doing a one-off, you should raise an issue with the JS community that having it would be great (and if you can implement it, even better!)
  • if we ever have such a documentation on manual context propagation, we should note that people do their due diligence first, i.e. check if an instrumentation library exists & then dive into doing it manually.

svrnm avatar Oct 17 '22 07:10 svrnm

Thanks for your response @svrnm , I have been able to pull this off. This is what I've done

In the calling function,

const opentelemetry = require("@opentelemetry/api");
   const {W3CTraceContextPropagator}= require('@opentelemetry/core');

   const {traceId, spanId}= span['_spanContext'];
   const traceparent= `00-${traceId}-${spanId}-01`;
   opentelemetry.propagation.inject(opentelemetry.context.active());`

sent this traceparent in request header.

In the called service,

console.log('req headers: ', req.headers);
  const carrier= {
    "traceparent": req.headers.traceparent
  }
  const incomingContext= propagator.extract(opentelemetry.context.active(), carrier, opentelemetry.defaultTextMapGetter)
  
  const span = tracer.startSpan('Issuing Certificate By admin', undefined, incomingContext);

This worked for me. Please excuse the bad code formatting. Thanks a lot for the lead. Cheers

psk001 avatar Oct 20 '22 17:10 psk001