fluentd
fluentd copied to clipboard
fluent
Support for OTLP metrics export
Is your feature request related to a problem? Please describe.
Most modern observability tools use to push metrics instead of letting a server pulling them. It's less prone to error since who is generating the metrics can decide when to send metrics. Also, when we have fluentd running in containers, it's even harder to keep track of all instances and collect metrics from all of them. Having them sending metrics to a centralized gateway on the other side is much easier. On top of that, OpenTelemetry is becoming a standard in the industry and it seems to make sense having support for it in FluentD.
Describe the solution you'd like
Having a way of sending OTLP metrics to a configured OTLP endpoint.
Describe alternatives you've considered
The Prometheus plugin is very versatile and the OpenTelemetry collector has a scraper for prometheus endpoints, making it easy to initially migrate to OTel. But it doesn't solve the problem of having FluentD containerized and collecting metrics from all containers. Enabling FluentD to send those metrics periodically to a gateway seems more reliable to me.
Additional context
No response
In Fluent Bit side, it will be handled by cmetrics library. If we will support OTEL handling, we have to use and depend on it.
If we add cmetrics gem as runtime dependency, we can handle OTEL and Prometheus context natively in Fluentd core.
I would love to see this feature in place! Not sure how stable the cmetrics library is, but it seems like a good option.
Or, using otel official Ruby gem would be much better: https://github.com/open-telemetry/opentelemetry-ruby