gogstash icon indicating copy to clipboard operation
gogstash copied to clipboard

Do you have any plan to add plain codec?

Open yjagdale opened this issue 2 years ago • 5 comments

Do you have any plan to add plain codec?

yjagdale avatar Oct 08 '22 18:10 yjagdale

Hi

Could you tell a bit more about what you are trying to do? What inputs/outputs you want to use a plain codec on?

I assume that with "plain" you mean that a text string should not be handled in the input/output module.

helgeolav avatar Oct 08 '22 21:10 helgeolav

so right now every input which is coming from the source is converted to json (timestamp). In my case, I don't want that conversion and input -> filter it should be the same string as the source.

yjagdale avatar Oct 09 '22 05:10 yjagdale

https://www.elastic.co/guide/en/logstash/current/plugins-codecs-plain.html is the original codec from elastic.

yjagdale avatar Oct 09 '22 05:10 yjagdale

Ok, can you tell me what inputs and outputs you work with?

An event contains several fields. Timestamp and message are the two most used.

An input without a codec will typically place the input data into the field message. If the same input is configured to use the json codec, the input data will be parsed and new fields will be created for each set of data in the json.

A timestamp is always added to the event on input.

The filters work on whatever field you want it to work on.

On the output side it is somewhat similar. The elastic output will always convert the event into a JSON struct, meaning all fields, including the timestamp, will be included. Other outputs can handle this differently, allowing you to either choose a codec (json vs plain text), or what field to push out.

helgeolav avatar Oct 09 '22 11:10 helgeolav

The filters grok and gonx can be used instead of a codec to parse lines of text.

helgeolav avatar Apr 29 '23 10:04 helgeolav