fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

out_gelf compresses when chunking even if compress is set to false

Open jszwedko opened this issue 1 year ago • 0 comments

Bug Report

Describe the bug

The out_gelf plugin seems to compress packets when chunking even if compress is set to false.

To Reproduce

fluent.conf

[INPUT]
    Name                    tail
    Tag                     log.*
    Path                    /tmp/logs/*.log
    DB                      /tmp/fluent/flb_kube.db
    Mem_Buf_Limit           5MB
    Refresh_Interval        10
    Parser                  json

[OUTPUT]
    Name                    gelf
    Match                   log.*
    Host                    127.0.0.1
    Port                    15514
    Mode                    udp
    Gelf_Short_Message_Key  message
    Compress                false
    Packet_Size             5

[OUTPUT]
    Name                    stdout
    Match                   log.*

parser.conf

[PARSER]
    Name        json
    Format      json
    Time_Key    timestamp
    Time_Format %Y-%m-%dT%H:%M:%S %z

use netcat -ul 15514 | tee /tmp/capture.log

Write the following log to a file in /tmp/logs:

{"host": "localhost", "level": "debug", "message": "1", "time": "2006-07-28T13:22:04Z"}

Observe that the captured bytes are gzip'd.

This behavior is pretty clearly implemented in the code:

https://github.com/fluent/fluent-bit/blob/d94857f43ec2ff2741a0bdf7acd65a796c9132b5/plugins/out_gelf/gelf.c#L209-L226

Maybe this is intentional, but it is unexpected.

Expected behavior

Since Compress is set to false, chunked packets would not be compressed.

Screenshots

Your Environment

  • Version used: Fluent Bit v3.1.8
  • Configuration: above
  • Environment name and version (e.g. Kubernetes? What version?): none
  • Server type and version: none
  • Operating System and version: Darwin XXXX 23.6.0 Darwin Kernel Version 23.6.0: Wed Jul 31 20:49:39 PDT 2024; root:xnu-10063.141.1.700.5~1/RELEASE_ARM64_T6000 arm64 arm Darwin
  • Filters and plugins: in config

Additional context

I ran into this using fluent-bit to test a feature being added to Vector to decode chunked GELF messages: https://github.com/vectordotdev/vector/pull/20859

jszwedko avatar Oct 10 '24 23:10 jszwedko