opentelemetry-collector-contrib
opentelemetry-collector-contrib copied to clipboard
Collector not working when k8sattributes in use
Component(s)
processor/k8sattributes
What happened?
Description
I am trying to add k8sattributes
to a gateway collector, but the collector and health check are not functioning. The collector appears to start, but refuses connections on the receiving ports. The health check endpoint returns a 503 with {"status":"Server not available","upSince":"0001-01-01T00:00:00Z","uptime":""}
.
Steps to Reproduce
- Build an image with the configuration below.
- Run it.
Expected Result
- Collector responds to health checks at
curl -v http://localhost:13133
. - Received traces include k8s attributes.
Actual Result
The collector never becomes healthy, and does not accept any signals.
Collector version
0.111.0
Environment information
Environment
OS: macOS 15.0.1 (Docker), and GKE Autopilot
OpenTelemetry Collector configuration
extensions:
health_check:
endpoint: "0.0.0.0:13133"
receivers:
otlp:
protocols:
http:
endpoint: "0.0.0.0:4318"
include_metadata: true
processors:
k8sattributes:
auth_type: "serviceAccount"
extract:
metadata:
# - k8s.namespace.name
# - k8s.deployment.name
# - k8s.statefulset.name
# - k8s.daemonset.name
# - k8s.cronjob.name
# - k8s.job.name
# - k8s.node.name
- k8s.pod.name
# - k8s.pod.uid
# - k8s.pod.start_time
passthrough: false
# pod_association:
# - sources:
# - from: resource_attribute
# name: k8s.pod.ip
# - sources:
# - from: resource_attribute
# name: k8s.pod.uid
# - sources:
# - from: connection
exporters:
# NOTE: Add this to the list of pipeline exporters to see the collector's debug logs
debug:
verbosity: detailed
service:
extensions: [ health_check ]
# https://opentelemetry.io/docs/collector/configuration/#telemetry
telemetry:
# This controls log verbosity of the collector itself.
logs:
encoding: json
level: "debug"
pipelines:
traces:
receivers: [ otlp ]
processors: [ k8sattributes ]
exporters: [ debug ]
Log output
otel-collector-1 | {"level":"info","ts":1729289848.7740626,"caller":"[email protected]/service.go:136","msg":"Setting up own telemetry..."}
otel-collector-1 | {"level":"info","ts":1729289848.7742362,"caller":"telemetry/metrics.go:70","msg":"Serving metrics","address":"localhost:8888","metrics level":"Normal"}
otel-collector-1 | {"level":"info","ts":1729289848.7743702,"caller":"builders/builders.go:26","msg":"Development component. May change in the future.","kind":"exporter","data_type":"traces","name":"debug"}
otel-collector-1 | {"level":"debug","ts":1729289848.774658,"caller":"builders/builders.go:24","msg":"Beta component. May change in the future.","kind":"processor","name":"k8sattributes","pipeline":"traces"}
otel-collector-1 | {"level":"debug","ts":1729289848.774745,"caller":"builders/builders.go:24","msg":"Stable component.","kind":"receiver","name":"otlp","data_type":"traces"}
otel-collector-1 | {"level":"debug","ts":1729289848.7748547,"caller":"builders/extension.go:48","msg":"Beta component. May change in the future.","kind":"extension","name":"health_check"}
otel-collector-1 | {"level":"info","ts":1729289848.7753859,"caller":"[email protected]/service.go:208","msg":"Starting otelcol-contrib...","Version":"0.111.0","NumCPU":16}
otel-collector-1 | {"level":"info","ts":1729289848.775412,"caller":"extensions/extensions.go:39","msg":"Starting extensions..."}
otel-collector-1 | {"level":"info","ts":1729289848.775442,"caller":"extensions/extensions.go:42","msg":"Extension is starting...","kind":"extension","name":"health_check"}
otel-collector-1 | {"level":"info","ts":1729289848.7754776,"caller":"[email protected]/healthcheckextension.go:33","msg":"Starting health_check extension","kind":"extension","name":"health_check","config":{"Endpoint":"0.0.0.0:13133","TLSSetting":null,"CORS":null,"Auth":null,"MaxRequestBodySize":0,"IncludeMetadata":false,"ResponseHeaders":null,"CompressionAlgorithms":null,"ReadTimeout":0,"ReadHeaderTimeout":0,"WriteTimeout":0,"IdleTimeout":0,"Path":"/","ResponseBody":null,"CheckCollectorPipeline":{"Enabled":false,"Interval":"5m","ExporterFailureThreshold":5}}}
otel-collector-1 | {"level":"warn","ts":1729289848.7757652,"caller":"[email protected]/warning.go:40","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks.","kind":"extension","name":"health_check","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
otel-collector-1 | {"level":"info","ts":1729289848.7758508,"caller":"extensions/extensions.go:59","msg":"Extension started.","kind":"extension","name":"health_check"}
### Additional context
I run `curl -v http://localhost:13133` to check if the collector is healthy.