logstash-filter-csv icon indicating copy to clipboard operation
logstash-filter-csv copied to clipboard

[Docs] improvement request for skip_header, autodetect_column_names setting

Open dharada opened this issue 5 years ago • 1 comments

This is not feature request, but Docs improvement request by a user.

  • Version: 3.0.10

https://www.elastic.co/guide/en/logstash/current/plugins-filters-csv.html#plugins-filters-csv-skip_header

Define whether the header should be skipped. Defaults to false. 
If set to true, the header will be skipped. Assumes that header is not repeated within further rows as such rows will also be skipped. 
If skip_header is set without autodetect_column_names being set then columns should be set which will result in the skipping of any row that exactly matches the specified column values. 
If skip_header and autodetect_column_names are specified then columns should not be specified, 
in this case autodetect_column_names will fill the columns setting in the background, from the first event seen, and any subsequent values that match what was autodetected will be skipped.

As current official document says, we can confirm below behavior.

product behavior

When both skip_header and autodetect_column_names are specified as true, autodetect_column_names uses the first event for auto-generate the column names, and first event is skipped by skip_header(true). so csv filtering process is taken from second event which doesn't match first event.

a suggestion

How about adding above product behavior as a clear example ? (if this make documents more understandable.)

dharada avatar Dec 02 '19 11:12 dharada

Also on the same topic: https://github.com/logstash-plugins/logstash-filter-csv/issues/79

AndyHunt66 avatar Feb 19 '20 10:02 AndyHunt66