fluent-operator
fluent-operator copied to clipboard
fluent
Add a new way to deploy fluentbit
Is your feature request related to a problem? Please describe.
Since fluentbit upgraded to 1.9, many plugins have been added, such as the Prometheus Scrape Metrics plugin, but it is not convenient to collect some information if configured in the form of daemonset. So I would like to add the statefulset method to deploy fluentbit.
Describe the solution you'd like
To do this we need to add a new CRD, our CRD based on fluentbit's statefulset can be called collector.
Describe alternatives you've considered
To do this we may also need to modify the fluentbit-controller.
Additional context
No response
How do you grab all pods logs if FB is not deployed as Daemonset ? FB reads files on each nodes to collect pods logs. If you have FB on only some K8s nodes, you'll be missing plenty of pod logs.
How do you grab all pods logs if FB is not deployed as Daemonset ? FB reads files on each nodes to collect pods logs. If you have FB on only some K8s nodes, you'll be missing plenty of pod logs. @jcdauchy-moodys
Fluent Operator support deploying fluent bit as a daemonset already. What we're talking about in this issue is the requirement to deploy fluentbit as a statefulset to scrap Prometheus metrics which doesn't require to be deployed as a daemonset
Add FluentBit collector is helpful in scenarios of collecting data through the network including the following input plugins:
- https://docs.fluentbit.io/manual/pipeline/inputs/opentelemetry
- https://docs.fluentbit.io/manual/pipeline/inputs/prometheus-scrape-metrics
- https://docs.fluentbit.io/manual/pipeline/inputs/collectd
- https://docs.fluentbit.io/manual/pipeline/inputs/forward
- https://docs.fluentbit.io/manual/pipeline/inputs/http
- https://docs.fluentbit.io/manual/pipeline/inputs/mqtt
- https://docs.fluentbit.io/manual/pipeline/inputs/nginx
- https://docs.fluentbit.io/manual/pipeline/inputs/statsd
- https://docs.fluentbit.io/manual/pipeline/inputs/syslog
- https://docs.fluentbit.io/manual/pipeline/inputs/syslog
- https://docs.fluentbit.io/manual/pipeline/inputs/tcp
related PR: https://github.com/fluent/fluent-operator/pull/484