logstash-filter-csv icon indicating copy to clipboard operation
logstash-filter-csv copied to clipboard

Please allow for ruby's CSV 2.4.0 :liberal_parsing

Open Skeeve opened this issue 6 years ago • 0 comments

When you dump data from (e.g.) mysql (using "--batch" and "--raw") you get something CSV-like.

The issues here are

  1. Fields are not quoted
  2. Quotes can appear in values
  3. Tab is the separator

Ideally we would have the possibility to tell the underlying CSV class that we do not have quoted fields, that quotes need not be escaped and have a (more convenient) way to define a control-character as the separator.

But I think the most important issue is that we would need to be able to set :liberal _parsing

Cite:

When set to a true value, CSV will attempt to parse input not conformant with RFC 4180, such as double quotes in unquoted fields.

Skeeve avatar Feb 18 '19 07:02 Skeeve