telegraf
telegraf copied to clipboard
Published
20 hours ago •
influxdata
influxdata
telegraf inputs.influxdb_listener parser_type = "upstream" silently dropping metrics
Relevant telegraf.conf
[[inputs.influxdb_listener]]
service_address = ":8248"
read_timeout = "10s"
write_timeout = "10s"
max_body_size = 0
parser_type = "upstream"
[inputs.influxdb_listener.tags]
influxdb_bucket = "storage-customer"
[[outputs.influxdb_v2]]
urls = ["http://127.0.0.1:8086"]
token = "REDACTED"
organization = "REDACTED"
bucket = "storage-customer"
tagexclude = ["influxdb_bucket"]
[outputs.influxdb_v2.tagpass]
influxdb_bucket = ["storage-customer"]
Logs from Telegraf
Sep 6 16:42:00 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:00Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 144.707325ms
Sep 6 16:42:00 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:00Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:42:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:01Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 79.004685ms
Sep 6 16:42:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:01Z D! [outputs.influxdb_v2] Wrote batch of 788 metrics in 14.73283ms
Sep 6 16:42:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:02Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 57.578988ms
Sep 6 16:42:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:10 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:10Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 149.217257ms
Sep 6 16:42:10 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:10Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:42:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:11Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 126.093318ms
Sep 6 16:42:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:11Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:11Z D! [outputs.influxdb_v2] Wrote batch of 412 metrics in 9.177343ms
Sep 6 16:42:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:11Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:11Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:12Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 1.869867ms
Sep 6 16:42:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:20 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:20Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 151.708474ms
Sep 6 16:42:20 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:20Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:42:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:21Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 182.417027ms
Sep 6 16:42:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:21Z D! [outputs.influxdb_v2] Wrote batch of 413 metrics in 9.164516ms
Sep 6 16:42:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:22Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 2.139642ms
Sep 6 16:42:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:30 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:30Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 153.11613ms
Sep 6 16:42:30 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:30Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:42:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:31Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 204.769999ms
Sep 6 16:42:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:31Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:31Z D! [outputs.influxdb_v2] Wrote batch of 469 metrics in 11.934351ms
Sep 6 16:42:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:31Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:31Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:32Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 1.700305ms
Sep 6 16:42:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:40 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:40Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 151.251302ms
Sep 6 16:42:40 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:40Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:42:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:41Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 140.882303ms
Sep 6 16:42:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:41Z D! [outputs.influxdb_v2] Wrote batch of 471 metrics in 11.063073ms
Sep 6 16:42:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:42Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 1.732968ms
Sep 6 16:42:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:50 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:50Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 133.922589ms
Sep 6 16:42:50 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:50Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:42:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:51Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 129.918228ms
Sep 6 16:42:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:51Z D! [outputs.influxdb_v2] Wrote batch of 452 metrics in 9.867903ms
Sep 6 16:42:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:42:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:52Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 52.234692ms
Sep 6 16:42:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:42:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:00 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:00Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 217.276945ms
Sep 6 16:43:00 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:00Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:43:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:01Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 143.567922ms
Sep 6 16:43:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:01Z D! [outputs.influxdb_v2] Wrote batch of 784 metrics in 12.830987ms
Sep 6 16:43:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:02Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 73.32357ms
Sep 6 16:43:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:10 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:10Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 137.747641ms
Sep 6 16:43:10 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:10Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:43:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:11Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 283.442264ms
Sep 6 16:43:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:11Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:11Z D! [outputs.influxdb_v2] Wrote batch of 412 metrics in 11.124937ms
Sep 6 16:43:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:11Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:11Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:12Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 1.83124ms
Sep 6 16:43:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:20 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:20Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 136.510288ms
Sep 6 16:43:20 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:20Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:43:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:21Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 147.056788ms
Sep 6 16:43:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:21Z D! [outputs.influxdb_v2] Wrote batch of 419 metrics in 10.004562ms
Sep 6 16:43:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:22Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 23.106434ms
Sep 6 16:43:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:30 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:30Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 140.434547ms
Sep 6 16:43:30 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:30Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:43:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:31Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 97.589452ms
Sep 6 16:43:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:31Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:31Z D! [outputs.influxdb_v2] Wrote batch of 453 metrics in 9.796021ms
Sep 6 16:43:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:31Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:31Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:32Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 22.134073ms
Sep 6 16:43:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:40 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:40Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 130.828462ms
Sep 6 16:43:40 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:40Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:43:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:41Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 68.995264ms
Sep 6 16:43:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:41Z D! [outputs.influxdb_v2] Wrote batch of 511 metrics in 11.440868ms
Sep 6 16:43:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:42Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 5.024934ms
Sep 6 16:43:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:50 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:50Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 124.910515ms
Sep 6 16:43:50 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:50Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:43:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:51Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 152.377221ms
Sep 6 16:43:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:51Z D! [outputs.influxdb_v2] Wrote batch of 468 metrics in 10.713938ms
Sep 6 16:43:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:43:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:52Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 2.570475ms
Sep 6 16:43:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:43:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:00 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:00Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 154.809293ms
Sep 6 16:44:00 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:00Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:44:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:01Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 129.039107ms
Sep 6 16:44:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:01Z D! [outputs.influxdb_v2] Wrote batch of 792 metrics in 13.73905ms
Sep 6 16:44:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:01 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:01Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:02Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 13.973655ms
Sep 6 16:44:02 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:02Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:10 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:10Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 152.35592ms
Sep 6 16:44:10 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:10Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:44:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:11Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 138.830157ms
Sep 6 16:44:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:11Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:11Z D! [outputs.influxdb_v2] Wrote batch of 408 metrics in 10.518448ms
Sep 6 16:44:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:11Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:11 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:11Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:12Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 1.800418ms
Sep 6 16:44:12 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:12Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:20 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:20Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 152.704663ms
Sep 6 16:44:20 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:20Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:44:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:21Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 129.504526ms
Sep 6 16:44:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:21Z D! [outputs.influxdb_v2] Wrote batch of 409 metrics in 10.744654ms
Sep 6 16:44:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:21 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:21Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:22Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 1.879567ms
Sep 6 16:44:22 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:22Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:30 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:30Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 137.857095ms
Sep 6 16:44:30 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:30Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:44:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:31Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 208.60908ms
Sep 6 16:44:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:31Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:31Z D! [outputs.influxdb_v2] Wrote batch of 461 metrics in 12.681482ms
Sep 6 16:44:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:31Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:31 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:31Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:32Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 1.920263ms
Sep 6 16:44:32 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:32Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:40 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:40Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 140.556086ms
Sep 6 16:44:40 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:40Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:44:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:41Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 109.037353ms
Sep 6 16:44:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:41Z D! [outputs.influxdb_v2] Wrote batch of 495 metrics in 11.665845ms
Sep 6 16:44:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:41 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:41Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:42Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 1.840292ms
Sep 6 16:44:42 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:42Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:50 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:50Z D! [outputs.influxdb_v2] Wrote batch of 10000 metrics in 132.71962ms
Sep 6 16:44:50 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:50Z D! [outputs.influxdb_v2] Buffer fullness: 3731 / 100000 metrics
Sep 6 16:44:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:51Z D! [outputs.influxdb_v2] Wrote batch of 3731 metrics in 137.570303ms
Sep 6 16:44:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:51Z D! [outputs.influxdb_v2] Wrote batch of 464 metrics in 10.874654ms
Sep 6 16:44:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:51 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:51Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
Sep 6 16:44:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:52Z D! [outputs.influxdb_v2] Wrote batch of 11 metrics in 1.793536ms
Sep 6 16:44:52 tuk6r2influxdbstorage2 telegraf[53689]: 2022-09-06T16:44:52Z D! [outputs.influxdb_v2] Buffer fullness: 0 / 100000 metrics
System info
Ubuntu 18.04, telegraf 1.23.3-1, influxdb 2.4.0-1
Docker
No response
Steps to reproduce
- Include
parser_type = "upstream"and restart telegraf - metrics aren't written to the bucket,
- remove parser_type = "upstream" and restart telegraf
- Metrics are written to the bucket
(see the two screenshot of telegraf metrics before and after the write showing that it appears the data is written
...
Expected behavior
data is written to the bucket
Actual behavior
no data is written to the bucket
Additional info
I can't find any influxdb logs or metrics that would indicate the the metrics were dropped. Metrics to many other buckets are not dropped.
The code that writes this is a very straightforward use of the influx_client.
influx_client = DataFrameClient(args.influx_url.hostname, args.influx_url.port, database=args.influx_db)
fields = ['physical', 'virtual']
tags = ['name', 'email', 'region', 'poolset_key', 'account_key']
report_df = pd.read_csv(args.tsvfile, sep='\t', comment='#')
date_today = datetime.now(timezone.utc)
report_df['date'] = pd.to_datetime(date_today)
report_df = report_df.set_index('date')
influx_client.create_database(args.influx_db)
influx_client.write_points(report_df, args.measurement_name, time_precision='m', tag_columns=tags, field_columns=fields)
```
I had recently debugged another issue with the upsteam parser @powersj https://github.com/influxdata/telegraf/issues/10900
Hi @andrewbierbaum,
Per the Telegraf logs you provided, it appears that 1000s of metrics are getting read and sent to InfluxDB. Are the numbers in the logs what you expect?
I am not sure I fully understand your screenshots. Were you expecting to see internal metrics with much larger numbers of requests and writes?
@powersj
Sorry, that wasn't clear. There are 6 different buckets on that host that are correctly injesting metrics using parser_type = "upstream" - the logs show correctly show a lot of clean writes. I'm also seeing metrics flow into correctly into many other buckets using parser_type = "upstream" on other hosts as well. However, this one bucket for reasons I'm not clear on silently drops metrics when using parser_type = "upstream", and writes metrics when that flag is removed.
The screenshot shows that telegraf records that it's cleanly writing into that bucket, but no new metics are actually appearing in that bucket.
Hi @andrewbierbaum,
Thanks for the clarification, but it is not clear to me where metrics are getting lost based on these comments:
this one bucket for reasons I'm not clear on silently drops metrics screenshot shows that telegraf records that it's cleanly writing into that bucket
The first sounds like Telegraf is dropping the metrics. The second sounds like metrics are written, but not seen in the bucket.
Is it possible to share some of the metrics that the bad bucket is writing? Could you remove the "upstream" parser config option and output to using the outputs.file output? Then do the same with the upstream parser and see if a tag or metrics themselves are getting dropped along the way?
@powersj Sorry for the delay, main project got busy for a few days. I added the following file output to the above telegraf I had used.
[[outputs.file]]
files = ["/tmp/metrics.out"]
data_format = "influx"
influx_max_line_bytes = 0
tagexclude = ["influxdb_bucket"]
[outputs.file.tagpass]
influxdb_bucket = ["storage-customer"]
without using the upsteam parser, the timestamps look good, and metrics write cleanly
18:53 andrewbierbaum@tuk6r2influxdbstorage2:~$ head metrics_no_parser.out
used,account_key=account-2,[email protected],host=tuk6r2influxdbstorage2,name=Time\ Lords,poolset_key=skytap-shared,region=tuk6r2 physical=0.41,virtual=6.62 1663180320000000000
used,account_key=account-c-0a72d3f5b4ccb3fa,[email protected],host=tuk6r2influxdbstorage2,name=FieldOps\ -\ Mayank\ Test,poolset_key=skytap-shared,region=tuk6r2 physical=0.06,virtual=0.29 1663180320000000000
used,account_key=account-c-c308c268e3333ff8,[email protected],host=tuk6r2influxdbstorage2,name=Costello\ Inc.,poolset_key=skytap-shared,region=tuk6r2 physical=0.05,virtual=0.21 1663180320000000000
used,account_key=account-c-57942ee14dec9419,[email protected],host=tuk6r2influxdbstorage2,name=Skytap\ Docs,poolset_key=skytap-shared,region=tuk6r2 physical=0.03,virtual=0.2 1663180320000000000
used,account_key=account-c-951cea3de8226afc,[email protected],host=tuk6r2influxdbstorage2,name=R2104\ (b24f),poolset_key=skytap-shared,region=tuk6r2 physical=0.03,virtual=0.12 1663180320000000000
when using the upstream parser, the timestamps are massively in the past, and telegraf records that it writes metrics, but influx should immediately trim them as being too old for the retention policy.
18:53 andrewbierbaum@tuk6r2influxdbstorage2:~$ head metrics_with_upstream_parser.out
used,account_key=account-2,[email protected],host=tuk6r2influxdbstorage2,name=Time\ Lords,poolset_key=skytap-shared,region=tuk6r2 physical=0.41,virtual=6.62 27719661
used,account_key=account-c-0a72d3f5b4ccb3fa,[email protected],host=tuk6r2influxdbstorage2,name=FieldOps\ -\ Mayank\ Test,poolset_key=skytap-shared,region=tuk6r2 physical=0.06,virtual=0.29 27719661
used,account_key=account-c-c308c268e3333ff8,[email protected],host=tuk6r2influxdbstorage2,name=Costello\ Inc.,poolset_key=skytap-shared,region=tuk6r2 physical=0.05,virtual=0.21 27719661
used,account_key=account-c-57942ee14dec9419,[email protected],host=tuk6r2influxdbstorage2,name=Skytap\ Docs,poolset_key=skytap-shared,region=tuk6r2 physical=0.03,virtual=0.2 27719661
used,account_key=account-c-951cea3de8226afc,[email protected],host=tuk6r2influxdbstorage2,name=R2104\ (b24f),poolset_key=skytap-shared,region=tuk6r2 physical=0.03,virtual=0.12 27719661
again, the only difference between these two tests was parser_type = "upstream" either not being present or being present in the telegraf config
Let me know if there is anything more I can do to diagnose this. Thanks!
@andrewbierbaum, Thanks for the update and now my time to apologize of the lack of response, I've been out of the office.
Can I ask what input plugin is used to collect this data? I would like to somehow reproduce this. I was thinking of using the example output data you have, but wanted to know how you are sending it to see if I can replicate.
@powersj it was produced from the following python snippet - using the python influxdb client version 5.3.1
influx_client = DataFrameClient(args.influx_url.hostname, args.influx_url.port, database=args.influx_db)
fields = ['physical', 'virtual']
tags = ['name', 'email', 'region', 'poolset_key', 'account_key']
report_df = pd.read_csv(args.tsvfile, sep='\t', comment='#')
date_today = datetime.now(timezone.utc)
report_df['date'] = pd.to_datetime(date_today)
report_df = report_df.set_index('date')
influx_client.create_database(args.influx_db)
influx_client.write_points(report_df, args.measurement_name, time_precision='m', tag_columns=tags, field_columns=fields)
writing to the following telegraf
[agent]
metric_batch_size = 10000
metric_buffer_limit = 100000
[[inputs.influxdb_listener]]
service_address = ":8248"
read_timeout = "10s"
write_timeout = "10s"
max_body_size = 0
parser_type = "upstream"
[inputs.influxdb_listener.tags]
influxdb_bucket = "storage-customer"
[[outputs.influxdb_v2]]
urls = ["http://127.0.0.1:8086"]
token = "REDACTED"
organization = "REDACTED"
bucket = "storage-customer"
tagexclude = ["influxdb_bucket"]
[outputs.influxdb_v2.tagpass]
influxdb_bucket = ["storage-customer"]
[[outputs.file]]
files = ["/tmp/metrics.out"]
data_format = "influx"
influx_max_line_bytes = 0
tagexclude = ["influxdb_bucket"]
[outputs.file.tagpass]
influxdb_bucket = ["storage-customer"]
and writing to influxdb 2.4.0
Thanks for the config and code! You have an interesting mix of InfluxDB v1 Python client library, writing to InfluxDB v1 listener, but writing out to the InfluxDB v2 output.
I was able reproduce locally and narrow it down. In your Python you are specifying the precision as a minute:
time_precision='m',
The new upstream parser library only accepts n, u, ms, and s. That similar switch is replicated in telegraf's parser logic. Unfortunately, we are not responding with any error to let you know either.
I was curious if this was an omission or what is actually supported it seems it is the consistent list of supported precision for all 2.0 related items:
I have put up #11866 to respond with a 400 and a message about unsupported precision values, h and m. I think for your use-case you have two ways forward: revert back to the internal parser or switch to a different precision if possible.
Thanks!