fluent-bit icon indicating copy to clipboard operation
fluent-bit copied to clipboard

Did Elasticsearch out plugin support compression yet?

Open april4 opened this issue 3 years ago • 3 comments

Is your feature request related to a problem? Please describe.

Current es out plugin seems not enabled compression, and also no params/option to controll that.
If sent out content not compressed, then Elasticsearch's response will also no compress, which will cause significant in/out bandwidth usage, especially when Fluent-bit and Elasticsearch are not in same region.

Describe the solution you'd like

Describe alternatives you've considered

Additional context

ref: https://github.com/fluent/fluent-bit/issues/3971

april4 avatar Jun 18 '22 03:06 april4

https://github.com/fluent/fluent-bit/pull/5557

patrick-stephens avatar Jun 18 '22 07:06 patrick-stephens

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

github-actions[bot] avatar Sep 17 '22 02:09 github-actions[bot]

It should now support compression (as of 1.9.7 IIRC), but the PR for documenting the feature has not been approved/merged yet.

rarruda avatar Sep 17 '22 07:09 rarruda

The (currently unmerged) PR with the documentation on how to enable compression can be found at: https://github.com/fluent/fluent-bit-docs/pull/824

rarruda avatar Sep 23 '22 05:09 rarruda

@april4 Short answer: yes, compress is now supported in the elasticsearch output.

See https://docs.fluentbit.io/manual/v/2.0-pre/pipeline/outputs/elasticsearch and compress parameter on how to configure it in the latest version of fluentbit.

rarruda avatar Oct 05 '22 22:10 rarruda

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days. Maintainers can add the exempt-stale label.

github-actions[bot] avatar Jan 04 '23 02:01 github-actions[bot]

This issue was closed because it has been stalled for 5 days with no activity.

github-actions[bot] avatar Jan 10 '23 02:01 github-actions[bot]