logstash-plugins
[ESF] Add ability to disable document enrichment
This is an ask from the Elastic Cloud Monitoring team, in order to enable Logstash output for our Elastic Serverless Forwarder product. https://github.com/elastic/elastic-serverless-forwarder/issues/112.
Based on discussions we have had already with the Logstash team, we have identified the HTTP input to be the most suitable way to send data from our AWS serverless collector to Logstash. The issue is that the plugin enriches input data with additional fields such as host.ip and http.version etc. We would like a way to disable this behaviour, leaving the input documents untouched.
A similar issue has been raised on the elastic agent plugin. https://github.com/logstash-plugins/logstash-input-elastic_agent/issues/3.
Working directly on this plugin, we can avoid adding additional metadata introducing a new config value, and use it as flag to add / not add the header medatata, together with event.original
We can define this flag as:
- An additional parameter to be written in logstash configs
- An entry that is added by ESF to the event, before sending it to Logstash
I personally prefer option 1 as I think it's closer to what an user would expect, and users would not need to add extra dummy data to their logs just for this purpose.
I am currently working on a PR to add this functionality https://github.com/logstash-plugins/logstash-input-http/pull/158
I think we can close this as not needed anymore :) (cc @yaauie @roaksoax @tommyers-elastic )