charts
charts copied to clipboard
Published
20 hours ago •
SigNoz
SigNoz
k8s_* metrics not showing in Digital Ocean K8s
kubeletstats receiver not working as expected.
Error log:
2023-02-22T06:53:05.880Z error [email protected]/scraper.go:80 call to /stats/summary endpoint failed {"kind": "receiver", "name": "kubeletstats", "pipeline": "metrics", "error": "Get \"http://pool-iytqfvim8-qo0cj:10255/stats/summary\": dial tcp: lookup pool-iytqfvim8-qo0cj on 10.245.0.10:53: no such host"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver.(*kubletScraper).scrape
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/scraper.go:80
go.opentelemetry.io/collector/receiver/scraperhelper.ScrapeFunc.Scrape
go.opentelemetry.io/[email protected]/receiver/scraperhelper/scraper.go:31
go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport
go.opentelemetry.io/[email protected]/receiver/scraperhelper/scrapercontroller.go:211
go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1
go.opentelemetry.io/[email protected]/receiver/scraperhelper/scrapercontroller.go:193
Same problem here
Digital Oean K8s
The IP from the error is kube-dns svc:
kube-system kube-dns ClusterIP 10.245.0.10 dns:53►0╱UDP dns-tcp:53►0 metrics:9153►0
Errors logs:
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x 2023-09-06T22:25:02.668Z error scraperhelper/scrapercontroller.go:213 Error scraping metrics {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics", "error": "Get \"https://w-pool-uy4qaitgx-y74ag:10250/stats/summary\": dial tcp: lookup w-pool-uy4qaitgx-y74ag on 10.245.0.10:53: no such host", "scraper": "kubeletstats"} │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:213 │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1 │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:192 │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x 2023-09-06T22:25:32.665Z error [email protected]/scraper.go:68 call to /stats/summary endpoint failed {"kind": "receiver", "name": "kubeletstats", "data_type": "metrics", "error": "Get \"https://w-pool-uy4qaitgx-y74ag:10250/stats/summary\": dial tcp: lookup w-pool-uy4qaitgx-y74ag on 10.245.0.10:53: no such host"} │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x github.com/open-telemetry/opentelemetry-collector-contrib/receiver/kubeletstatsreceiver.(*kubletScraper).scrape │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/scraper.go:68 │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/receiver/scraperhelper.ScrapeFunc.Scrape │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/[email protected]/scraperhelper/scraper.go:20 │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).scrapeMetricsAndReport │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:210 │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/receiver/scraperhelper.(*controller).startScraping.func1 │
│ signoz-k8s-metrics-exporter-k8s-infra-otel-agent-dnl9x go.opentelemetry.io/collector/[email protected]/scraperhelper/scrapercontroller.go:192
Tracking issue: https://github.com/open-telemetry/opentelemetry-collector-contrib/issues/22843
@paulohmorais Meanwhile, you can use cAdvisor with otel-collector-metrics instead of kubeletstats receiver. As mentioned: https://github.com/SigNoz/dashboards/issues/13#issuecomment-1589036731