opentelemetry-cpp-contrib icon indicating copy to clipboard operation
opentelemetry-cpp-contrib copied to clipboard

Httpd traces missing service name

Open jvilhuber opened this issue 3 years ago • 7 comments

Describe your environment Describe any aspect of your environment relevant to the problem, including your platform, build system, version numbers of installed dependencies, etc. If you're reporting a problem with a specific version of a library in this repo, please check whether the problem has been fixed on main branch.

Running a compiled-from-source apache httpd 2.4.51 (latest), with the downloaded mod-otel (0.1.0):

Configs:

OpenTelemetryExporter   otlp
OpenTelemetryEndpoint otel-collector.default.svc.cluster.local:4317
OpenTelemetryPropagators trace-context
OpenTelemetryIgnoreInbound off
OpenTelemetrySetResource service.name test.service

I configured the otel-collector to push traces to a grafana-tempo instance as well as a jaeger server:

receivers:
  jaeger:                            # the receives all come from the OpenTelemetry collector.  more configuration information can
    protocols:                       # be found there: https://github.com/open-telemetry/opentelemetry-collector/tree/main/receiver
      thrift_http:                   #
      grpc:                          # for a production deployment you should only enable the receivers you need!
      thrift_binary:
      thrift_compact:
  otlp:
    protocols:
      grpc:
      http:
processors:
  batch:
extensions:
  health_check:
  pprof:
  zpages:
exporters:
  otlp:
    endpoint: tempo-otlp.observability.example.com:443
  jaeger:
    endpoint: "jaeger.example.com:14250"
    tls:
      insecure: true
service:
  extensions: [health_check,pprof,zpages]
  pipelines:
    traces/1:
      receivers: [otlp,jaeger]
      processors: [batch]
      exporters: [otlp,jaeger]
  telemetry:
    logs:
      level: "info"
  1. In tempo, the traces seem to exist but show absolutely no data at all.
  2. In jaeger, The trace shows up, but jaeger says Service: OTLPResourceNoServiceName

Side-note: I can't tell what resource the otel module is sending, as the build doesn't seem to use the later opentelemetry-cpp library (1.0.0) which should include https://github.com/open-telemetry/opentelemetry-cpp/pull/906 So it's hard to say what's going on here. As far as I can tell (newbie to telemetry!) service.name resource is required, but I suspect it's not being sent.

Steps to reproduce Describe exactly how to reproduce the error. Include a code sample if applicable.

apache-http 2.4.51 + mod-otel 0.1.0 -> otel-collector (otlp) -> tempo(otlp) and or jaeger

Given the right opentelemetry-cpp ostream output (see library version 1.0.0 and the PR above) this should be testable via the instrumentation/http/tests in some way.

What is the expected behavior? What did you expect to see?

I expected to see valid traces in both grafana-tempo and jaeger.

What is the actual behavior? What did you see instead?

"empty" traces in tempo (no data at all) and traces with OTLPResourceNoServiceName as the service name in jaeger.

Additional context Add any other context about the problem here.

jvilhuber avatar Dec 10 '21 06:12 jvilhuber

I found a similar (same?) bug in the nginx module: https://github.com/open-telemetry/opentelemetry-cpp-contrib/issues/27 Perhaps this just needs a bump to the latest opentelemetry-cpp library?

jvilhuber avatar Dec 10 '21 07:12 jvilhuber

@TomRoSystems Any chance you could update the library to something recent to see if this fixes some of the issues I've had? I suspect https://github.com/open-telemetry/opentelemetry-cpp-contrib/issues/78 is also related to this.

jvilhuber avatar Dec 13 '21 11:12 jvilhuber

@TomRoSystems Any update on this one?

jvilhuber avatar Jan 18 '22 07:01 jvilhuber

FWIW I did a local test upgrading to opentelemetry-cpp-1.0.0-rc1 (the latest version that builds without Api changes) and it seems to fix the problem. So I'm pretty sure just upgrading the opentelemetry-cpp library will do the trick here. I got stuck on adding a default service.name, however, and since cpp isn't my language, I'm not sure any PR of mine would be good enough (though I'm happy to share what little I did i.e. hacked up).

jvilhuber avatar Jan 18 '22 07:01 jvilhuber

Are you building with Bazel or CMake? As I think CMake was already using version you mentioned - setup-cmake.sh line 62

TomRoSystems avatar Jan 18 '22 17:01 TomRoSystems

I was using bazel, but cmake seems faster, so I can use that instead. Any chance you can upgrade to the latest opentelemetry (1.1.0 I think)? EDIT: How are the assets under https://github.com/open-telemetry/opentelemetry-cpp-contrib/releases built? bazel or cmake?

jvilhuber avatar Jan 19 '22 02:01 jvilhuber