fluent-operator icon indicating copy to clipboard operation
fluent-operator copied to clipboard

add emitterMemBufLimit and emitterStorageType to rewriteTag

Open chrono2002 opened this issue 1 year ago • 0 comments

Input pauses on high pressure in complicated configs (using rewriteTag).

"If the destination for the new records generated might face backpressure due to latency or slow network, we strongly recommend enabling the filesystem mode." (https://docs.fluentbit.io/manual/pipeline/filters/rewrite-tag)

It's better to have an ability to change emitter parameters to safe ones.

Add emitter parameters to rewriteTAg and multiline, eg:

  - rewriteTag:
      emitterName: rewrite_em
      emitterStorageType: filesystem
      emitterMemBufLimit: 128MB
 
  - multiline:
      emitterName: multiline_em
      emitterStorageType: filesystem
      emitterMemBufLimit: 128MB

See this PR for a possible solution: https://github.com/fluent/fluent-operator/pull/1069

chrono2002 avatar Feb 28 '24 18:02 chrono2002