fluent-operator
fluent-operator copied to clipboard
fluent
Fluentd ： Failed to add out elasticsearch authentication
==================== apiVersion: v1 data: password: cEBzc3cwcmQ= username: ZWxhc3RpYw== kind: Secret metadata: name: es-secret namespace: logging type: Opaque
apiVersion: fluentd.fluent.io/v1alpha1 kind: ClusterOutput metadata: name: k8s-kube-system labels: output.fluentd.fluent.io/enabled: "true" output.fluentd.fluent.io/role: "log-operator" spec: outputs:
- elasticsearch: user: valueFrom: secretKeyRef: name: es-secret key: username password: valueFrom: secretKeyRef: name: es-secret key: password host: log-es.logging.svc port: 9200 logstashFormat: true logstashPrefix: k8s-kube-system
=======================
/ $ cat fluentd/etc/app.conf
===============
[ClusterFluentdConfig-cluster-fluentd-config-kube-system::cluster::clusteroutput::k8s-kube-system-0] Could not communicate to Elasticsearch, resetting connection and trying again. [401] {"error":{"root_cause":[{"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}}],"type":"security_exception","reason":"unable to authenticate user [elastic] for REST request [/]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}},"status":401} 2022-03-20 08:05:16 +0000 [warn]: #0 [ClusterFluentdConfig-cluster-fluentd-config-kube-system::cluster::clusteroutput::k8s-kube-system-0] Remaining retry: 9. Retry to communicate after 64 second(s)
===============
How to solve this change? There is absolutely nothing wrong with the password
This looks like there is a problem with validation. As you can see from the error message, the information about the user has been written to the configuration file.
thanks for reminding But I decrypt via base64 and the password is correct It looks like there is a problem with the secretKeyRef key, it gets my username
In my opinion, you can check this in two ways. First you can start by looking at the secret fluentd-config to check that the passwd is written correctly. Then you can check es to see if it can be accessed via that password, if not, change es.
The configuration file should not be placed in the deployment, it is the output crd. Here https://github.com/fluent/fluent-operator/blob/master/apis/fluentd/v1alpha1/plugins/output/es.go#L26.
One more thing, fluent-operator will generate the appropriate fluentd config file based on the crd you write and then mount it on fluentd, so if that doesn't work you can show me the secret fluentd-config. it's base64 encoded, you can decode it and see.
I tried to decode the secrets fluentd-config and found that the password it got was wrong
But the configuration in my ClusterOutput is correct
what's going on thanks
You can try creating two secrets and see if its passwd is the desired value.
This is a good idea But it doesn't solve the problem, it gets the username
Dears , After I modified it, I still can't push the data to es. What is the es version supported by fluentd? My es version is 7.12.0, but fluentbit can
Yes. It's an alarm, but I'm not sure if it will have other effects in subsequent use Is there a compatibility list?
Yes. It's an alarm, but I'm not sure if it will have other effects in subsequent use Is there a compatibility list?
Maybe needs a research between fluentd and elaticsearch v7.12.0. Would you like to implement this guide or give a pr on it?
It's okay, I just want to make sure the version is compatible. Of course, in my environment, the version of es is not a hard requirement I'm going to look up the compatibility of fluentd thank you very much
It's okay, I just want to make sure the version is compatible. Of course, in my environment, the version of es is not a hard requirement I'm going to look up the compatibility of fluentd thank you very much
No pls, it's an honor to accept your PR. If you have other questions, please let us know.
Ref: https://github.com/fluent/fluent-operator/blob/master/cmd/fluent-watcher/fluentd/Dockerfile.amd64. And we would do a research also.
The prompt of fluent bit log is as follows:
"2022/03/28 09:00:19] [ warn] [engine] chunk '16-1648458003.156515404.flb' cannot be retried: task_id=3, input=systemd.1 > output=es.0 [2022/03/28 09:00:19] [error] [output:es:es.0] HTTP status=401 URI=/_bulk, response: {"error":{"root_cause":[{"type":"security_exception","reason":"missing authentication credentials for REST request [/_bulk]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF- 8""}}],"type":"security_exception","reason":"missing authentication credentials for REST request [/_bulk]","header":{"WWW-Authenticate":"Basic realm="security" charset="UTF-8""}},"status":401}"
Hello, excuse me Elasticsearch sets user and password authentication. How does fluent bit authenticate?
@benjaminhuo @wenchajun有没有案例或解决方案？
kubectl create secret generic es-secret --from-literal=username=xxx --from-literal=password=xxx
ClusterOutput Yaml Fragment
spec:
outputs:
- elasticsearch:
host: xxx
logstashFormat: true
logstashPrefix: xxx
password:
valueFrom:
secretKeyRef:
key: password
name: es-secret
port: 9200
user:
valueFrom:
secretKeyRef:
key: username
name: es-secret