amazon-kinesis-firehose-for-fluent-bit icon indicating copy to clipboard operation
amazon-kinesis-firehose-for-fluent-bit copied to clipboard

replace_dots option for high performance version

Open kingsleykumar opened this issue 4 years ago • 4 comments

Hi team, I am planning to use this plugin in our production cluster. After going through the documentation, it looks like there is a new high performance version available, but it lacks the replace_dots option. I require that option to avoid some mapping conflicts with existing elasticsearch index. I would like to know when will the high performance version have the replace_dots option?

Thanks.

kingsleykumar avatar Aug 11 '21 14:08 kingsleykumar

I have added this on our backlog, but unfortunately our backlog is pretty long. Not certain when it will get done.

PettitWesley avatar Aug 16 '21 02:08 PettitWesley

@PettitWesley It's been more than an year now and this feature hasn't been implemented. Is there any workaround for this?

varun1231 avatar Nov 03 '22 15:11 varun1231

@varun1231 I would try seeing if there is a way to implement this using a lua filter or with the new web assembly filters:

  • https://docs.fluentbit.io/manual/pipeline/filters/lua
  • https://docs.fluentbit.io/manual/pipeline/filters/wasm

And may be also search upstream to see if anyone has created a script or solution and put it online.

PettitWesley avatar Nov 04 '22 00:11 PettitWesley

@PettitWesley It is surprising that such a functionality that has been included in elasticsearch and fluent bit go plugin isn't available in the fluent bit high performance plugin yet.

Why isn't this a priority? It is a well-known issue that comes up in the container traffic with kubernetes labels that has elasticsearch as it's final destination.

varun1231 avatar Nov 14 '22 13:11 varun1231