node-csv icon indicating copy to clipboard operation
node-csv copied to clipboard

`parse` with `{raw: true}` and windows line endings loses the newline

Open cakoose opened this issue 2 years ago • 0 comments

Describe the bug

I'm calling parse with {raw: true} to get the original line. When lines end in just "\n", things work fine. When lines end in "\r\n", the raw output is missing the final "\n".

To Reproduce

const {parse} = require('csv-parse');

const inputs = [
    'a,b\nc,d\n',
    'a,b\r\nc,d\r\n',
    'a,b\r\nc,d\n',
];

for (const input of inputs) {
    parse(input, {raw: true}, (err, data) => {
        if (err) {
            console.error(err);
            return;
        }
        console.log(data);
    });
}

Output:

[
  { record: [ 'a', 'b' ], raw: 'a,b\n' },
  { record: [ 'c', 'd' ], raw: 'c,d\n' }
]
[
  { record: [ 'a', 'b' ], raw: 'a,b\r' },
  { record: [ 'c', 'd' ], raw: 'c,d\r' }
]
[
  { record: [ 'a', 'b' ], raw: 'a,b\r' },
  { record: [ 'c', 'd\n' ], raw: 'c,d\n' }
]

First output: correct.

Second output: the raw fields are missing "\n" at the end. What I'd expect:

[
  { record: [ 'a', 'b' ], raw: 'a,b\r\n' },
  { record: [ 'c', 'd' ], raw: 'c,d\r\n' }
]

Third output: In addition to the missing "\n" in the first raw, the 'd\n' cell value is surprising! Mixed line endings are tricky, so maybe there's no 100% correct answer here. But mixed line endings do come up in practice, and I think this might be better:

[
  { record: [ 'a', 'b' ], raw: 'a,b\r\n' },
  { record: [ 'c', 'd' ], raw: 'c,d\n' }
]

Additional context

My higher-level goal: I'm trying to add a column of cells to an existing CSV. I'd like to avoid modifying the format of any of the existing cells, since subtle changes in quoting can change how tools like Excel interpret a CSV. I was hoping the {raw: true} option would give me a way to do that, but I ran into this issue.

cakoose avatar Apr 05 '22 02:04 cakoose