helm-charts icon indicating copy to clipboard operation
helm-charts copied to clipboard

Published 20 hours ago • verified publisher icon fluent

[fluentd] Detected ES 8.x or above: document type will not be used

Open willzhang opened this issue 1 year ago • 1 comments

install elasticsearch

helm repo add bitnami https //charts.bitnami.com/bitnami
helm install elasticsearch bitnami/elasticsearch \
  -n elasticsearch --create-namespace \
  --version=8.10.3

install fluentd

helm repo add fluent https://fluent.github.io/helm-charts
helm install fluentd fluent/fluentd -n fluent --create-namespace

pod start failed

root@ubuntu:~# kubectl -n fluent get pods
NAME            READY   STATUS             RESTARTS        AGE
fluentd-m6z6b   0/1     CrashLoopBackOff   6 (2m38s ago)   7m13s

what i changed in configmap

  <source>
    exclude_path /var/log/containers/*fluentd*.log
    <parse>
      @type "cri"
    </parse>
  </source>

error logs

root@ubuntu:~# kubectl -n fluent logs -f ds/fluentd
2023-10-13 08:06:45 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2023-10-13 08:06:45 +0000 [info]: parsing config file is succeeded path="/fluentd/etc/../../../etc/fluent/fluent.conf"
2023-10-13 08:06:46 +0000 [info]: gem 'fluentd' version '1.16.2'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-concat' version '2.5.0'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-dedot_filter' version '1.0.0'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-detect-exceptions' version '0.0.15'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-elasticsearch' version '5.2.5'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-grok-parser' version '2.6.2'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-json-in-json-2' version '1.0.2'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-kubernetes_metadata_filter' version '3.2.0'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-multi-format-parser' version '1.0.0'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-parser-cri' version '0.1.1'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-prometheus' version '2.1.0'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-record-modifier' version '2.1.1'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.4.0'
2023-10-13 08:06:46 +0000 [info]: gem 'fluent-plugin-systemd' version '1.0.5'
2023-10-13 08:06:46 +0000 [warn]: [filter_kube_metadata] !! The environment variable 'K8S_NODE_NAME' is not set to the node name which can affect the API server and watch efficiency !!
2023-10-13 08:06:46 +0000 [debug]: 'host elasticsearch.elasticsearch' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host: elasticsearch.elasticsearch' doesn't have tag placeholder
2023-10-13 08:06:46 +0000 [debug]: 'index_name fluentd' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'index_name: fluentd' doesn't have tag placeholder
2023-10-13 08:06:46 +0000 [debug]: 'template_name ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'template_name: ' doesn't have tag placeholder
2023-10-13 08:06:46 +0000 [debug]: 'logstash_prefix logstash' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_prefix: logstash' doesn't have tag placeholder
2023-10-13 08:06:46 +0000 [debug]: 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' has timestamp placeholders, but chunk key 'time' is not configured
2023-10-13 08:06:46 +0000 [debug]: 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' doesn't have tag placeholder
2023-10-13 08:06:46 +0000 [debug]: 'deflector_alias ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'deflector_alias: ' doesn't have tag placeholder
2023-10-13 08:06:46 +0000 [debug]: 'application_name default' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'application_name: default' doesn't have tag placeholder
2023-10-13 08:06:46 +0000 [debug]: 'ilm_policy_id logstash-policy' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'ilm_policy_id: logstash-policy' doesn't have tag placeholder
2023-10-13 08:06:46 +0000 [debug]: Need substitution: false
2023-10-13 08:06:46 +0000 [debug]: 'host_placeholder elasticsearch.elasticsearch' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host_placeholder: elasticsearch.elasticsearch' doesn't have tag placeholder
2023-10-13 08:06:46 +0000 [info]: using configuration file: <ROOT>
  <label @FLUENT_LOG>
    <match **>
      @type null
      @id ignore_fluent_logs
    </match>
  </label>
  <source>
    @type tail
    @id in_tail_container_logs
    @label @KUBERNETES
    path "/var/log/containers/*.log"
    exclude_path /var/log/containers/*fluentd*.log
    pos_file "/var/log/fluentd-containers.log.pos"
    tag "kubernetes.*"
    read_from_head true
    <parse>
      @type "cri"
    </parse>
  </source>
  <label @KUBERNETES>
    <match kubernetes.var.log.containers.fluentd**>
      @type relabel
      @label @FLUENT_LOG
    </match>
    <filter kubernetes.**>
      @type kubernetes_metadata
      @id filter_kube_metadata
      skip_labels false
      skip_container_metadata false
      skip_namespace_metadata true
      skip_master_url true
    </filter>
    <match **>
      @type relabel
      @label @DISPATCH
    </match>
  </label>
  <label @DISPATCH>
    <filter **>
      @type prometheus
      <metric>
        name fluentd_input_status_num_records_total
        type counter
        desc The total number of incoming records
        <labels>
          tag ${tag}
          hostname ${hostname}
        </labels>
      </metric>
    </filter>
    <match **>
      @type relabel
      @label @OUTPUT
    </match>
  </label>
  <label @OUTPUT>
    <match **>
      @type elasticsearch
      @log_level "debug"
      host "elasticsearch.elasticsearch"
      port 9200
    </match>
  </label>
</ROOT>
2023-10-13 08:06:46 +0000 [info]: starting fluentd-1.16.2 pid=7 ruby="3.1.4"
2023-10-13 08:06:46 +0000 [info]: spawn command to main:  cmdline=["/usr/local/bin/ruby", "-Eascii-8bit:ascii-8bit", "/fluentd/vendor/bundle/ruby/3.1.0/bin/fluentd", "-c", "/fluentd/etc/../../../etc/fluent/fluent.conf", "-p", "/fluentd/plugins", "--gemfile", "/fluentd/Gemfile", "-r", "/fluentd/vendor/bundle/ruby/3.1.0/gems/fluent-plugin-elasticsearch-5.2.5/lib/fluent/plugin/elasticsearch_simple_sniffer.rb", "--under-supervisor"]
2023-10-13 08:06:47 +0000 [info]: #0 init worker0 logger path=nil rotate_age=nil rotate_size=nil
2023-10-13 08:06:47 +0000 [info]: adding match in @FLUENT_LOG pattern="**" type="null"
2023-10-13 08:06:47 +0000 [info]: adding match in @KUBERNETES pattern="kubernetes.var.log.containers.fluentd**" type="relabel"
2023-10-13 08:06:47 +0000 [info]: adding filter in @KUBERNETES pattern="kubernetes.**" type="kubernetes_metadata"
2023-10-13 08:06:48 +0000 [warn]: #0 [filter_kube_metadata] !! The environment variable 'K8S_NODE_NAME' is not set to the node name which can affect the API server and watch efficiency !!
2023-10-13 08:06:48 +0000 [info]: adding match in @KUBERNETES pattern="**" type="relabel"
2023-10-13 08:06:48 +0000 [info]: adding filter in @DISPATCH pattern="**" type="prometheus"
2023-10-13 08:06:48 +0000 [info]: adding match in @DISPATCH pattern="**" type="relabel"
2023-10-13 08:06:48 +0000 [info]: adding match in @OUTPUT pattern="**" type="elasticsearch"
2023-10-13 08:06:48 +0000 [debug]: #0 'host elasticsearch.elasticsearch' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host: elasticsearch.elasticsearch' doesn't have tag placeholder
2023-10-13 08:06:48 +0000 [debug]: #0 'index_name fluentd' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'index_name: fluentd' doesn't have tag placeholder
2023-10-13 08:06:48 +0000 [debug]: #0 'template_name ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'template_name: ' doesn't have tag placeholder
2023-10-13 08:06:48 +0000 [debug]: #0 'logstash_prefix logstash' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_prefix: logstash' doesn't have tag placeholder
2023-10-13 08:06:48 +0000 [debug]: #0 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' has timestamp placeholders, but chunk key 'time' is not configured
2023-10-13 08:06:48 +0000 [debug]: #0 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' doesn't have tag placeholder
2023-10-13 08:06:48 +0000 [debug]: #0 'deflector_alias ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'deflector_alias: ' doesn't have tag placeholder
2023-10-13 08:06:48 +0000 [debug]: #0 'application_name default' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'application_name: default' doesn't have tag placeholder
2023-10-13 08:06:48 +0000 [debug]: #0 'ilm_policy_id logstash-policy' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'ilm_policy_id: logstash-policy' doesn't have tag placeholder
2023-10-13 08:06:48 +0000 [debug]: #0 Need substitution: false
2023-10-13 08:06:48 +0000 [debug]: #0 'host_placeholder elasticsearch.elasticsearch' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host_placeholder: elasticsearch.elasticsearch' doesn't have tag placeholder
2023-10-13 08:06:48 +0000 [debug]: #0 Detected ES 8.x or above: This parameter has no effect.
2023-10-13 08:06:48 +0000 [info]: adding source type="tail"
2023-10-13 08:06:48 +0000 [info]: #0 starting fluentd worker pid=18 ppid=7 worker=0
2023-10-13 08:06:48 +0000 [debug]: #0 buffer started instance=51640 stage_size=0 queue_size=0
2023-10-13 08:06:48 +0000 [debug]: #0 flush_thread actually running
2023-10-13 08:06:48 +0000 [debug]: #0 enqueue_thread actually running
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-apiserver-696d8c446b-n2j6w_calico-apiserver_calico-apiserver-8a186dfcee95866ca18461e3808e6eeec01c7223d7eac5f7f4868a99ada44c0b.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-apiserver-696d8c446b-n2j6w_calico-apiserver_calico-apiserver-b3e2663a5ac3b7173b2c777571219e312201c479ff157a95b5506567b33de89a.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-apiserver-696d8c446b-xfd44_calico-apiserver_calico-apiserver-69819c66e9e96b75189becb965e5b8103ad533326d64841330eda217be77390b.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-apiserver-696d8c446b-xfd44_calico-apiserver_calico-apiserver-e3681f131d36cc551f8bc61437f9f97b50a9a91d9c581684e062ea3b7ba8eed0.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-kube-controllers-9569d4577-6vpsp_calico-system_calico-kube-controllers-3592d873788749968c0d6b6bdb54657e5ee638752fff8ce3c77c6cf8ca2f6f98.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-kube-controllers-9569d4577-6vpsp_calico-system_calico-kube-controllers-f7297fa4d6c7886f723a7a0f9d36b15cf26121262c3b03cca057906a84b18b6e.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-node-8wbmt_calico-system_calico-node-1fb407ac4efe0b8ff22f067bc743cbfe666c0bfe058140c452cef3ba5416c142.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-node-8wbmt_calico-system_calico-node-6134c9ea58e783cb700020262cfcc4dcef27c942561c8a315d447f9ddd582e08.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-node-8wbmt_calico-system_flexvol-driver-9277bd154a76fd8cf8074467e62cc82de109c4a0c36eb2ecd16ebd2a48e4d4e0.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-node-8wbmt_calico-system_install-cni-ce44abcd75b2d73b02841db3144ce273335cbc3fae027244771e62ab691b35a6.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-typha-7dd44f9657-jncr6_calico-system_calico-typha-5c26c00c4b8c25d0b0c0793b284d7c452a6a7b7a9251193150a05718e9154ac3.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/calico-typha-7dd44f9657-jncr6_calico-system_calico-typha-bb74fa92b1dfe5c30e49c6d72fc9c8ba02865a27e5e8b308eaef7eae1ec2115a.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/coredns-5d78c9869d-d7gzq_kube-system_coredns-2de71f962dca3d23dd35c15040164f35e89631f38af07a8223eb1cd6928777b8.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/coredns-5d78c9869d-d7gzq_kube-system_coredns-5ed364850ea37e9140cb07bdab33ce3c713a21f4da87ac0a359775f95c343f98.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/coredns-5d78c9869d-j49sw_kube-system_coredns-7b0466ad7b85532909e81f6c3106cc405da598cb5fff8fad52cb14a2a8a8119c.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/coredns-5d78c9869d-j49sw_kube-system_coredns-be128f445c4542c5927f96b6a3dca60816c35a8eb3e0974897190d65419c4242.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/csi-node-driver-9mnc9_calico-system_calico-csi-78f0a2f7decb631c4c13f3b1606044caf8290209f69b6d2b0f820669021f3ac2.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/csi-node-driver-9mnc9_calico-system_calico-csi-c23001366d970323846dc99881294a8251f3722c2a7b91a29cd6e8bcc21253ae.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/csi-node-driver-9mnc9_calico-system_csi-node-driver-registrar-387df0c2c3ebd00eb6ed87c060c41a13322ead4b707b464e4cd954a0bdaf0267.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/csi-node-driver-9mnc9_calico-system_csi-node-driver-registrar-5b70c6ffd12daee157c9b9e6f6c9e8bf32817a6a7d67d841f344942a23b7d3d4.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-coordinating-0_elasticsearch_elasticsearch-bd73976716a8eb912910739e389951590a6fe0bb27ff8e3a41f8937b352b1114.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-coordinating-0_elasticsearch_sysctl-b381f8aa6acd374f801a5158d1a6e85fc8432126b26f0fa5a9ff41069b0b5fd4.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-coordinating-1_elasticsearch_elasticsearch-a6fab767cc055fe53dbbd44588ed81e934a0568eacc9b164e1b9d13f8be076c9.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-coordinating-1_elasticsearch_sysctl-798cb91ab6616ff847d5288b45044b09cc5a6ed11440cc2c43523dc48522bad4.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-data-0_elasticsearch_elasticsearch-73fbd4d939ab04c67ebda317633cf6aca2d97a4a2de704af84eeaccae161517d.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-data-0_elasticsearch_sysctl-183a59fbecf50bfa548845932d0b39e04f204c68e679d4b81a1b62e357572954.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-data-1_elasticsearch_elasticsearch-1228d16bf9edebf0bb27484685c4914a59457ef198d97645723fed8a2917cc35.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-data-1_elasticsearch_sysctl-c8ec5315040a17d8c3226e9dd58f46b27d8e571057072ae61dd16688e252e4a4.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-ingest-0_elasticsearch_elasticsearch-a962fc5014d360757d2e4268df01f5a3a64a8e10033514725a89e710c5f52cda.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-ingest-0_elasticsearch_sysctl-829a94b0a595876a62314b84853a4810df91c09f6fa070ada70225da8d1179c1.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-ingest-1_elasticsearch_elasticsearch-bc7502e240088ad7b177b0cc36863ae5a53ec2b420cb8acc5149035bbdf6302a.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-ingest-1_elasticsearch_sysctl-79507680e6bd91f64a532be6d58008d699dc011a2822afac8f9fb54cb06ef51e.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-master-0_elasticsearch_elasticsearch-1d18f46216cd0ec578f9fe7c577066630e5bbaa15a34df464b4128a067b99ddc.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-master-0_elasticsearch_sysctl-e5e3862420e21a8f06b363c4556dd036d7f0f93b687308fcac2f479a6cc47333.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-master-1_elasticsearch_elasticsearch-3e686a3ea5c0d3e8f648efec4eb9f3bad9b4f4f5df062696a3734203c6865b2f.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/elasticsearch-master-1_elasticsearch_sysctl-a3df0638ddb8651d47e9a4391231c4a01e615ea3dec67130b5f2fa7cbbba2f90.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/etcd-ubuntu_kube-system_etcd-93adf66d768421eb070dc0e1c1eb95404cb47d677ee519cbd1ecc4b1729173ca.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/etcd-ubuntu_kube-system_etcd-c99b2de5f751fb1887e0df7acf32b8248fc2db6046cd9335e81bf8675e681505.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kibana-79f59756f7-7dkvx_kibana_kibana-6469df00b3eedf151fdbac0c292bc289b7e8838bbf8e6d2f2ab3d1876f9a6acb.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kibana-79f59756f7-7dkvx_kibana_kibana-f064059bb2e1fa1fa3de0a58b1e3d71d197f08697c8157eb1545985c7ea84935.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kube-apiserver-ubuntu_kube-system_kube-apiserver-1e34e7fb843de23779b67279c64594ee57b69a8438ce18aa32f5830cdf9f6695.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kube-apiserver-ubuntu_kube-system_kube-apiserver-3759b23338accf6cf6ae63965fb201dc80eb44d8136f32591815f1d18a0b7722.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kube-controller-manager-ubuntu_kube-system_kube-controller-manager-378851add52214402cecc09fb2a975612da2893325f97b761cac4a9fdb7f5f93.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kube-controller-manager-ubuntu_kube-system_kube-controller-manager-d44cf142bf4882a669ad98d7c98f42f10baddefcf48b50d4ea74b096610306a9.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kube-proxy-m2qhh_kube-system_kube-proxy-1e51ffde91e75ccba5d46054d5ea1b38c738f7cfc39e192e06f7a5acfd94ce53.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kube-proxy-m2qhh_kube-system_kube-proxy-d0221969db9ed71963aba1e778cc55f39896b82b59ed88a9549cbf92438ec932.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kube-scheduler-ubuntu_kube-system_kube-scheduler-085d3f439c5367e0143a50e23f147f77e0b5fad73408e896c0ea6dd067d40028.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/kube-scheduler-ubuntu_kube-system_kube-scheduler-c0c333a5bde3ee5237f0397679ac8dbf4641fbc2ae3f628bfd5779727d0a3774.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/openebs-localpv-provisioner-7f5997cfb4-tzw7r_openebs_openebs-localpv-provisioner-12254975116358eebbf19db8f9657ac326ed5e7e1f78164955725c68c8e9e7a3.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/openebs-localpv-provisioner-7f5997cfb4-tzw7r_openebs_openebs-localpv-provisioner-4ad440e5c8926653621e14a102cd425322fa0486d35ecc339406a08f4841d2c0.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/tigera-operator-5f4668786-xxqhl_tigera-operator_tigera-operator-6d1eb9d0ff1248b502e92f650fcd88565f224e74706497959da87fcabd4aa633.log
2023-10-13 08:06:48 +0000 [info]: #0 [in_tail_container_logs] following tail of /var/log/containers/tigera-operator-5f4668786-xxqhl_tigera-operator_tigera-operator-7c3917725cba75e5a45fed7761687f4fd8a052f0731d9007b8fcc576dfd16b26.log
2023-10-13 08:06:48 +0000 [info]: #0 fluentd worker is now running worker=0
2023-10-13 08:07:14 +0000 [info]: Received graceful stop
2023-10-13 08:07:15 +0000 [info]: #0 fluentd worker is now stopping worker=0
2023-10-13 08:07:15 +0000 [info]: #0 shutting down fluentd worker worker=0
2023-10-13 08:07:15 +0000 [info]: #0 shutting down input plugin type=:tail plugin_id="in_tail_container_logs"
2023-10-13 08:07:15 +0000 [info]: #0 shutting down output plugin type=:null plugin_id="ignore_fluent_logs"
2023-10-13 08:07:15 +0000 [info]: #0 shutting down output plugin type=:relabel plugin_id="object:80c"
2023-10-13 08:07:15 +0000 [info]: #0 shutting down output plugin type=:relabel plugin_id="object:c8c8"
2023-10-13 08:07:15 +0000 [info]: #0 shutting down filter plugin type=:kubernetes_metadata plugin_id="filter_kube_metadata"
2023-10-13 08:07:15 +0000 [info]: #0 shutting down output plugin type=:relabel plugin_id="object:c990"
2023-10-13 08:07:15 +0000 [info]: #0 shutting down filter plugin type=:prometheus plugin_id="object:c97c"
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:15 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:16 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:17 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:17 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:18 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:19 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:19 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:20 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:20 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:20 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:20 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:21 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:21 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:21 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:21 +0000 [warn]: #0 dump an error event: error_class=Fluent::Plugin::ElasticsearchErrorHandler::ElasticsearchError error="400 - Rejected by Elasticsearch [error type]: document_parsing_exception [reason]: '[1:683] object mapping for [kubernetes.labels.app] tried to parse field [app] as object, but found a concrete value'" location=nil tag="kubernetes.var.log.containers.kibana-79f59756f7-7dkvx_kibana_kibana-f064059bb2e1fa1fa3de0a58b1e3d71d197f08697c8157eb1545985c7ea84935.log" time=2023-10-13 08:05:50.822011862 +0000 record={"stream"=>"stdout", "logtag"=>"F", "message"=>"[2023-10-13T08:05:50.821+00:00][INFO ][plugins.fleet] Running Fleet Usage telemetry send task", "time"=>"2023-10-13T16:05:50.822011862+08:00", "docker"=>{"container_id"=>"f064059bb2e1fa1fa3de0a58b1e3d71d197f08697c8157eb1545985c7ea84935"}, "kubernetes"=>{"container_name"=>"kibana", "namespace_name"=>"kibana", "pod_name"=>"kibana-79f59756f7-7dkvx", "container_image"=>"sealos.hub:5000/bitnami/kibana:8.10.2-debian-11-r11", "container_image_id"=>"sealos.hub:5000/bitnami/kibana@sha256:1b72dff7ba7ea0211d2fd1e1c764b0bc58a859ea450d2e283a5586f1020fba22", "pod_id"=>"1f3da37a-844b-4768-aba0-fd736fdc3c63", "pod_ip"=>"100.86.125.51", "host"=>"ubuntu", "labels"=>{"app"=>"kibana", "app.kubernetes.io/instance"=>"kibana", "app.kubernetes.io/managed-by"=>"Helm", "app.kubernetes.io/name"=>"kibana", "app.kubernetes.io/version"=>"8.10.2", "helm.sh/chart"=>"kibana-10.5.6", "pod-template-hash"=>"79f59756f7"}}, "@timestamp"=>"2023-10-13T08:05:50.822011862+00:00", "tag"=>"kubernetes.var.log.containers.kibana-79f59756f7-7dkvx_kibana_kibana-f064059bb2e1fa1fa3de0a58b1e3d71d197f08697c8157eb1545985c7ea84935.log"}
2023-10-13 08:07:21 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:21 +0000 [warn]: #0 dump an error event: error_class=Fluent::Plugin::ElasticsearchErrorHandler::ElasticsearchError error="400 - Rejected by Elasticsearch [error type]: document_parsing_exception [reason]: '[1:976] object mapping for [kubernetes.labels.app] tried to parse field [app] as object, but found a concrete value'" location=nil tag="kubernetes.var.log.containers.kibana-79f59756f7-7dkvx_kibana_kibana-f064059bb2e1fa1fa3de0a58b1e3d71d197f08697c8157eb1545985c7ea84935.log" time=2023-10-13 08:05:56.847339234 +0000 record={"stream"=>"stdout", "logtag"=>"F", "message"=>"[2023-10-13T08:05:56.846+00:00][INFO ][plugins.fleet] Fleet Usage: {\"agents_enabled\":true,\"agents\":{\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"inactive\":0,\"unenrolled\":0,\"total_all_statuses\":0,\"updating\":0},\"fleet_server\":{\"total_all_statuses\":0,\"total_enrolled\":0,\"healthy\":0,\"unhealthy\":0,\"offline\":0,\"updating\":0,\"num_host_urls\":0}}", "time"=>"2023-10-13T16:05:56.847339234+08:00", "docker"=>{"container_id"=>"f064059bb2e1fa1fa3de0a58b1e3d71d197f08697c8157eb1545985c7ea84935"}, "kubernetes"=>{"container_name"=>"kibana", "namespace_name"=>"kibana", "pod_name"=>"kibana-79f59756f7-7dkvx", "container_image"=>"sealos.hub:5000/bitnami/kibana:8.10.2-debian-11-r11", "container_image_id"=>"sealos.hub:5000/bitnami/kibana@sha256:1b72dff7ba7ea0211d2fd1e1c764b0bc58a859ea450d2e283a5586f1020fba22", "pod_id"=>"1f3da37a-844b-4768-aba0-fd736fdc3c63", "pod_ip"=>"100.86.125.51", "host"=>"ubuntu", "labels"=>{"app"=>"kibana", "app.kubernetes.io/instance"=>"kibana", "app.kubernetes.io/managed-by"=>"Helm", "app.kubernetes.io/name"=>"kibana", "app.kubernetes.io/version"=>"8.10.2", "helm.sh/chart"=>"kibana-10.5.6", "pod-template-hash"=>"79f59756f7"}}, "@timestamp"=>"2023-10-13T08:05:56.847339234+00:00", "tag"=>"kubernetes.var.log.containers.kibana-79f59756f7-7dkvx_kibana_kibana-f064059bb2e1fa1fa3de0a58b1e3d71d197f08697c8157eb1545985c7ea84935.log"}
2023-10-13 08:07:21 +0000 [debug]: #0 Indexed (op = index), 2 bad_argument
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:22 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:23 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:24 +0000 [debug]: #0 Detected ES 8.x or above: document type will not be used.
2023-10-13 08:07:25 +0000 [info]: #0 shutting down output plugin type=:elasticsearch plugin_id="object:c9a4"
2023-10-13 08:07:25 +0000 [debug]: #0 closing buffer instance=51640
2023-10-13 08:07:25 +0000 [info]: Worker 0 finished with status 0

willzhang avatar Oct 13 '23 08:10 willzhang

here is the fix:

https://github.com/fluent/helm-charts/issues/428

SlavikCA avatar Apr 18 '24 13:04 SlavikCA