jackson-dataformats-text
jackson-dataformats-text copied to clipboard
FasterXML
Headers seems to be trimmed
If I have a CSV with a header row, the header names seem to be trimmed. For example:
"Padded ,Unpadded\nOne,Two"
The above will produce a map with the key "Padded" instead of "Padded ".
Yes, header names are trimmed using same rules as values. To retain spaces, it would be necessary to either double-quote contents, or use escape mechanism (if enabled via CsvSchema).
My understanding is that this is what CSV specification (loose as it is) suggests.
That's a bummer. AWS writes it's detailed billing files making Tags into headers. Tags can have leading or trailing space so " Header" != "Header " != "Header". The Apache CSV parser handles this correctly (as well as most python ones). I love the Jackson Parser but there seems to be no way to make it work for this case. I did code up a crazy solution of reading String[] and making the first row create a header Map<String,Index> and then fake it out into thinking it was iterating Map<String,String> instead of String[]. But it's a lot of code just to work around this issue.
I really believe that your interpretation of the CSV spec is correct. The problem is that no one else's is. Also a flag just to remove that hard coded trim would be pretty awesome.
There is CsvParser.Feature.TRIM_SPACES, which you can disable, although it affects headers and values the same way.
Would it also make sense to file an issue against AWS? (regardless of if Jackson can work with this). White-space that is not enclosed in quotes (or escaped) really is not properly handled when writing CSV, so they are producing something that does not really inter-operate well. This assuming I understand your issue correctly.
I could file an issue with AWS but I doubt they'll fix it.
As far as I can tell disabling CsvParser.Feature.TRIM_SPACE will only fix values. Headers are always trimmed in CsvParser. It's hard coded.
https://github.com/FasterXML/jackson-dataformats-text/blob/939c7f949fa9d4e35f05c8c78e6cc09384ab1fd2/csv/src/main/java/com/fasterxml/jackson/dataformat/csv/CsvParser.java#L737
Actually it looks like even if it is enclosed in quotes that the above line will trim it.
Hmmh. Ok, that seems wrong then. I do think enclosed spaces should be retained.