fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

When "remove_keys" is added, all keys in content is cleared

Open duj4 opened this issue 11 months ago • 2 comments

Bug Report

Describe the bug No idea why all keys in log content is cleared.

To Reproduce

    [INPUT]
        Name tail
        Path /var/log/containers/kube*.log
        Path_Key filename
        Parser cri
        Tag kube.*
        Mem_Buf_Limit 5MB
        Skip_Long_Lines On
        Read_From_Head True
    [PARSER]
        # http://rubular.com/r/tjUt3Awgg4
        Name cri
        Format regex
        Regex ^(?<time>[^ ]+) (?<stream>stdout|stderr) (?<logtag>[^ ]*) (?<message>.*)$
        Time_Key    time
        Time_Format %Y-%m-%dT%H:%M:%S.%L%z
        Time_Keep   Off
    [FILTER]
        Name kubernetes
        Match kube.*
        Merge_Log On
        Keep_Log Off
        K8S-Logging.Parser On
        K8S-Logging.Exclude On
        labels Off
        annotations Off
    [FILTER]
        Name nest
        Match kube.*
        Operation lift
        Nested_under kubernetes
        Add_prefix kubernetes_
    [FILTER]
        Name modify
        Match kube.*
        Remove stream
        Remove logtag
    [OUTPUT]
        name                   loki
        match                  kube.*
        host                   loki-write.logging.svc.cluster.local
        port                   3100
        label_keys             $filename, $kubernetes_pod_name, $kubernetes_namespace_name, $kubernetes_pod_id, $kubernetes_host, $kubernetes_container_name, $kubernetes_docker_id, $kubernetes_container_hash, $kubernetes_container_image
        **remove_keys            kubernetes_pod_name** 
        drop_single_key        on
        line_format            key_value
        tenant_id              test6
        tls                    on
        tls.verify             on
        tls.debug              4
        tls.ca_file            /fluent-bit/etc/certs/ca.crt
        tls.crt_file           /fluent-bit/etc/certs/tls.crt
        tls.key_file           /fluent-bit/etc/certs/tls.key

Screenshots When "remove_keys" is not enabled: remove_no_keys

When "remove_keys" is enabled: remove_one_key

Your Environment

  • Version used: 2.2.2
  • Environment name and version (e.g. Kubernetes? What version?): Kubernetes
  • Server type and version: centos

duj4 avatar Mar 24 '24 12:03 duj4