fluent-operator
fluent-operator copied to clipboard
fluent
help request: `opentelemetry` input does not update the `fluent-bit` service
Describe the issue
Adding opentelemetry input/clusterinput resources neither triggers addition of 4318 port to existing fluent-bit service, nor creation of a new one. The fluentbit-config k8s secret is updated successfully:
fluent-bit.conf: |
[Service]
Http_Server true
Parsers_File /fluent-bit/etc/parsers.conf
[Input]
Name opentelemetry
listen 0.0.0.0
port 4318
tag_key traces
...
fluent-bit cotainers are successfully loading the configuration and starting listening on port 4318:
* Copyright (C) 2015-2024 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io
______ _ _ ______ _ _ _____ __
| ___| | | | | ___ (_) | |____ |/ |
| |_ | |_ _ ___ _ __ | |_ | |_/ /_| |_ __ __ / /`| |
| _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / \ \ | |
| | | | |_| | __/ | | | |_ | |_/ / | |_ \ V /.___/ /_| |_
\_| |_|\__,_|\___|_| |_|\__| \____/|_|\__| \_/ \____(_)___/
[2024/11/15 11:24:38] [ info] [fluent bit] version=3.1.7, commit=c6e902a43a, pid=12
[2024/11/15 11:24:38] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2024/11/15 11:24:38] [ info] [cmetrics] version=0.9.5
[2024/11/15 11:24:38] [ info] [ctraces ] version=0.5.5
[2024/11/15 11:24:38] [ info] [input:opentelemetry:opentelemetry.0] initializing
[2024/11/15 11:24:38] [ info] [input:opentelemetry:opentelemetry.0] storage_strategy='memory' (memory only)
[2024/11/15 11:24:38] [ info] [input:opentelemetry:opentelemetry.0] listening on 0.0.0.0:4318
...
but the pod and service ports configs are not updated, thus the port is not properly exposed
pod ⬇️
...
name: fluent-bit
ports:
- containerPort: 2020
name: metrics
protocol: TCP
resources:
...
service ⬇️
apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/name: fluent-bit
name: fluent-bit
...
spec:
ports:
- name: metrics
port: 2020
protocol: TCP
targetPort: 2020
selector:
app.kubernetes.io/name: fluent-bit
...
How did you install fluent operator?
i've used this chart https://github.com/fluent/fluent-operator/tree/master/charts/fluent-operator
Additional context
No response
You can check the logs of the fluent-operator to see if everything is normal.
here are the logs from the fluent-operator pod:
fluent-operator 2024-11-19T11:45:50Z INFO setup starting manager
setenv + echo 'CONTAINER_ROOT_DIR=/var/log'
fluent-operator 2024-11-19T11:45:50Z INFO controller-runtime.metrics Starting metrics server
fluent-operator 2024-11-19T11:45:50Z INFO controller-runtime.metrics Serving metrics server {"bindAddress": ":8080", "secure": false}
fluent-operator 2024-11-19T11:45:50Z INFO starting server {"name": "health probe", "addr": ":8081"}
Stream closed EOF for fluent/fluent-operator-57cb87b7cc-2gkp5 (setenv)
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.FluentBit"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1alpha1.Fluentd"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1.Secret"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.ClusterFluentBitConfig"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.FluentBitConfig"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.ClusterInput"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.ClusterFilter"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.ClusterOutput"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.ClusterParser"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.ClusterMultilineParser"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.Filter"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.Output"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.Parser"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.MultilineParser"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting Controller {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1.ServiceAccount"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1.DaemonSet"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1.StatefulSet"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1.Service"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting Controller {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1alpha1.Fluentd"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1.Secret"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1alpha1.ClusterFluentdConfig"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1alpha1.FluentdConfig"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1alpha1.Filter"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1alpha1.ClusterFilter"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1alpha1.Output"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "source": "kind source: *v1alpha1.ClusterOutput"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting Controller {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1alpha2.FluentBit"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1.ServiceAccount"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "source": "kind source: *v1.DaemonSet"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting Controller {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "collector", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "Collector", "source": "kind source: *v1alpha2.Collector"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "collector", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "Collector", "source": "kind source: *v1.ServiceAccount"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "collector", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "Collector", "source": "kind source: *v1.StatefulSet"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting EventSource {"controller": "collector", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "Collector", "source": "kind source: *v1.Service"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting Controller {"controller": "collector", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "Collector"}
fluent-operator 2024-11-19T11:45:56Z INFO Starting workers {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "worker count": 1}
fluent-operator 2024-11-19T11:45:56Z INFO Starting workers {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "worker count": 1}
fluent-operator 2024-11-19T11:45:56Z INFO Starting workers {"controller": "fluentbit", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "FluentBit", "worker count": 1}
fluent-operator 2024-11-19T11:45:56Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO Starting workers {"controller": "collector", "controllerGroup": "fluentbit.fluent.io", "controllerKind": "Collector", "worker count": 1}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO controllers.FluentBitConfig Fluent Bit main configuration has updated {"logging-control-plane": "fluent", "fluentbitconfig": "fluentbit-config", "secret": "fluentbit-config"}
fluent-operator 2024-11-19T11:45:57Z INFO Starting workers {"controller": "fluentd", "controllerGroup": "fluentd.fluent.io", "controllerKind": "Fluentd", "worker count": 1}