fluent-operator icon indicating copy to clipboard operation
fluent-operator copied to clipboard

Helm Chart: Add FluentBit Prometheus integration

Open hostalp opened this issue 2 years ago • 1 comments

Is your feature request related to a problem? Please describe.

Please add the Prometheus support into the Helm Chart just like the https://github.com/fluent/helm-charts/tree/main/charts/fluent-bit (for example) has. It enables the metrics HTTP server and creates a service resource for it, with optionally using the prometheus.io annotations (for the "old" style Prometheus server support), or servicemonitor (monitoring.coreos.com/v1) resource (for the Prometheus Operator support).

Describe the solution you'd like

Similarly to https://github.com/fluent/helm-charts/tree/main/charts/fluent-bit

  • ability to enable the FluentBit metrics HTTP server and create a Service resource for it (optionally with prometheus.io annotations to enable scraping)
  • ability to create the ServiceMonitor resource for the above service to enable discovery by the Prometheus Operator

Additionally

  • ability to enable the Grafana dashboard

Describe alternatives you've considered

As a workaround, one can currently try to modify the existing default ClusterFluentBitConfig resource to enable the metrics HTTP server (similarly to https://github.com/fluent/fluent-operator#monitoring), then create the required service and servicemonitor resources manually to get the Prometheus Operator integration to work. I haven't tried that however (but I believe it should work).

For the Grafana dashboard there isn't direct support, so only adding it externally would be an option.

Additional context

I don't currently use the Fluentd so the above is related to the FluentBit only. However something similar may be needed for the Fluentd as well if that one is also able to provide Prometheus metrics.

hostalp avatar Apr 21 '22 03:04 hostalp

Thanks for this feature request. I remember @benjaminhuo @wenchajun also mentioned the Prometheus metrics support for the Helm chart in the Fluent Operator roadmap. Maybe we could collaborate to make it happen.

FeynmanZhou avatar Apr 21 '22 03:04 FeynmanZhou

We have completed the relevant development, refer to this https://github.com/fluent/fluent-operator/blob/master/charts/fluent-operator/values.yaml#L122 I will close this issue and if you have any problems, you can reopen it.

wenchajun avatar Apr 11 '23 06:04 wenchajun