microservices-demo icon indicating copy to clipboard operation
microservices-demo copied to clipboard

Jaeger trace shows blank tags under Processes node

Open sundar-cs opened this issue 3 years ago • 4 comments

Original issue title: Jaeger trace shows blank tags under Processes node

This is just a query about using this sample app on an existing k8s cluster (on AWS for instance) (1) Save the YAML from this link into a local file. https://github.com/GoogleCloudPlatform/microservices-demo/blob/master/release/kubernetes-manifests.yaml (2) Uncomment the JAEGER_SERVICE_ADDR name and value in all the services (3) Create/Apply this YAML along with Jaeger (jaeger-all-in-one-template.yaml) onto the k8s cluster

The Jaeger UI (json view) shows the serviceName under Processes but the tags are always blank. Whereas another sample app such as HotRod (released as part of Jaeger) shows non-empty list of tags (such as ip, hostname etc).

Is this difference attributed to OpenCensus talking to Jaeger Collector directly whereas HotRod talks to Jaeger Agent (via OpenTracing libraries) which in turn relays it to the Jaeger Collector ? (with these tags getting inserted by the Agent or by the instrumentation code or library)

If so, what's the right way to generate the tags: would it be to route it via OpenCensus Agent ? Or is there a version of this microservices-app that works with OpenTracing ? (or OpenTelemetry)

Sample outputs are attached (trimmed versions). Thank you.

sample.hotrod.trimmed.json.txt sample.googleapp.json.trimmed.txt

sundar-cs avatar Aug 25 '20 11:08 sundar-cs

+1

sagarpatil23192 avatar Apr 21 '21 13:04 sagarpatil23192

To clarify, the normal trace tags are there but not the process tags correct?

If so, I've been able to reproduce this issue on GKE as well:

  • Create a GKE cluster;
  • Install the Jaeger operator with WATCH_NAMESPACE: "" (https://github.com/jaegertracing/jaeger-operator#getting-started);
  • Deploy Online Boutique with JAEGER_SERVICE_ADDR: "<name_of_jaeger_resource>-collector: 14268" (kubectl apply -f release/kubernetes-manifests.yaml);
  • Create a Jaeger resource in the default namespace (https://github.com/jaegertracing/jaeger-operator#getting-started)
  • Navigate to the newly created Jaeger Ingress' external IP and generate traces for any of the Online Boutique components;
  • Notice the "process" tags being grey'd out / empty: image

I spent some time troubleshooting this issue but can't seem to quite put my finger on why exactly the process tags aren't generated by the OpenCensus / Jaeger exporters. There is a way to append to the process tags manually (using JAEGER_TAGS: "key=value" but I don't think that is relevant to this issue, since we're looking for the pre-populated process tags (host, IP, version, etc.)

One thing to note is that the OpenCensus Jaeger exporters we are using hasn't been officially updated upstream since 2019 (e.g. https://github.com/census-ecosystem/opencensus-go-exporter-jaeger/releases). It is quite possible that an update may be needed to fix this process tags issue.

We are planning on eventually moving to OpenTelemetry, as OpenCensus is deprecating sometime in 2021. (https://github.com/GoogleCloudPlatform/microservices-demo/issues/422)

bourgeoisor avatar Apr 29 '21 18:04 bourgeoisor

Hello! Just a heads up that we're still aware of this issue, but fixing it is not high priority this quarter due to Jaeger being open-source and not a Google product. We will update when we are able to tackle this. Thanks!

askmeegs avatar Nov 19 '21 16:11 askmeegs

Commenting to reset OSLO-- I'm wondering if this might be fixed alongside https://github.com/GoogleCloudPlatform/microservices-demo/issues/422

bourgeoisor avatar Jun 21 '22 22:06 bourgeoisor

Closing this one as Jaeger has been removed from the project: https://github.com/GoogleCloudPlatform/microservices-demo/pull/1088, coming in 0.3.10.

mathieu-benoit avatar Oct 03 '22 14:10 mathieu-benoit