fluentd
fluentd copied to clipboard
Allow compressed data for tcp/udp/syslog inputs
Is your feature request related to a problem? Please describe.
To reduce network congestion, we want to enabled log compression when forwarding logs to our fluentd server. Fortunately, most of our tools can handle sending compressed syslog or raw data. Unfortunately, fluentd doesn't and takes the data "as-is". It appears in buffer files as strings containing only escaped unicode characters (ex: =\u0001\u0000
).
Describe the solution you'd like
It would be great if we could setup tcp, udp or syslog inputs to receive compressed data. Having a compress
directive accepting as values a boolean (true/false
) or a compression algorithm (none/gzip/...
).
<source>
@type tcp
port 3333
bind 0.0.0.0
tag input.myapp
compress gzip
<format>
@type none
</format>
</source>
Describe alternatives you've considered
So far we managed to activate compression on our data by logging our tools activities into files and by using a local td-agent tailing each file and forwarding their contents to our td-agent central server with compression enabled.
Additional context
No response