telegraf
telegraf copied to clipboard
influxdata
timestream output plugin with multi-measure records
Feature Request
Opening a feature request kicks off a discussion.
Proposal:
In last AWS ReInvent conference, multi-measure records where announced: https://youtu.be/QjMZaKkq8YM?t=1163 . This changes the format of data storage, from this:
| time | hostname | measure_name | measure_value |
|---|---|---|---|
| 2021-09-20 19:00:00 | host1 | cpu | 0.10 |
| 2021-09-20 19:00:00 | host1 | mem | 60 |
| 2021-09-20 19:00:00 | host1 | temp | 77 |
into this:
| time | host | cpu | mem | temp |
|---|---|---|---|---|
| 2021-09-20 19:00:00 | host1 | 0.10 | 60 | 77 |
With this new table approach, there is a significant storage reduction and queries on data are simpler.
There is more info here but I can't find technical documentation.
Should timestream output plugin be able to push multi-measure records?
Definitely!! Multi-record is an excellent option for many time series data and since TimeStream now supports it, Telegraf should also allow us to take advantage of it.
Accordind to AWS here TimeStream output plugin already supports this, but I cant's find this "multi-measure in single row" feature in any of the latest Telegraf versions.
Looking at the Go code, this version of Telegraf doesn't have multi-measure support. I really wish it did!
I believe we are waiting on some answers to https://github.com/influxdata/telegraf/pull/11385 before landing this feature.
