logstash icon indicating copy to clipboard operation
logstash copied to clipboard

Doc: Evaluate and action Beats module messaging and deprecation

Open karenzone opened this issue 1 year ago • 2 comments

Current offerings -- Elastic Agent, Elastic integrations, and the [logstash-filter-agent_integration](https://www.elastic.co/guide/en/logstash/master/ea-integrations.html -- might give users better ways to achieve the same results.

Phase 1: Evaluate recommendations

@flexitrev @jsvd @robbavey @karenzone

Filebeat modules

  • [ ] https://www.elastic.co/guide/en/logstash/master/filebeat-modules.html

Winlogbeat modules

  • [ ] https://www.elastic.co/guide/en/logstash/master/winlogbeat-modules.html

Phase 2: Implement recommendations

@karenzone et al

Related: #16590

karenzone avatar Oct 21 '24 14:10 karenzone

I think we should remove these pages and focus exclusively on https://www.elastic.co/guide/en/logstash/master/ea-integrations.html

flexitrev avatar Oct 21 '24 19:10 flexitrev

The most obvious answer here comes from the deprecation (or not) of these modules in 9.0.0. If there's messaging on the filebeat and winlogbeat that they're deprecated in 8.x and to be removed in 9.0.0 then we should reflect that messaging in the Logstash documentation side.

The documentation of integrating Logstash with Beats modules focuses on ensuring Logstash forwards the right ingest pipeline name to ES during the bulk request. The documentation mentioned by Trevor focuses on a different experience, running the ingest pipelines directly in Logstash.

There's also an architectural question here where we should likely be still recommending users to run the transformation in ES (by forwarding the pipeline or data stream name correctly) and only move to having run on Logstash if the user needs it (by configuring the elastic_integration filter).

jsvd avatar Oct 28 '24 09:10 jsvd