UnhandledPromiseRejectionWarning: TypeError [ERR_INVALID_ARG_TYPE]: The "list[1]" argument must be one of type Array, Buffer, or Uint8Array. Received type string

[braco]

Getting this error:

(node:99549) UnhandledPromiseRejectionWarning: TypeError [ERR_INVALID_ARG_TYPE]: The "list[1]" argument must be one of type Array, Buffer, or Uint8Array. Received type string
    at Function.concat (buffer.js:479:13)
    at concatLeftChunk (/project/node_modules/csvtojson/v2/dataClean.js:31:23)
    at Object.prepareData (/project/node_modules/csvtojson/v2/dataClean.js:14:21)
    at ProcessorLocal.process (/project/node_modules/csvtojson/v2/ProcessorLocal.js:86:37)
    at Converter._transform (/project/node_modules/csvtojson/v2/Converter.js:144:24)
    at Converter.Transform._read (/project/node_modules/readable-stream/lib/_stream_transform.js:184:10)
    at Converter.Readable.read (/project/node_modules/readable-stream/lib/_stream_readable.js:443:10)
    at flow (/project/node_modules/readable-stream/lib/_stream_readable.js:813:34)
    at DestroyableTransform.<anonymous> (/project/node_modules/readable-stream/lib/_stream_readable.js:683:7)
    at DestroyableTransform.emit (events.js:182:13)
    at onwriteDrain (/project/node_modules/readable-stream/lib/_stream_writable.js:501:12)
    at afterWrite (/project/node_modules/readable-stream/lib/_stream_writable.js:489:18)
    at onwrite (/project/node_modules/readable-stream/lib/_stream_writable.js:483:7)
    at WritableState.onwrite (/project/node_modules/readable-stream/lib/_stream_writable.js:180:5)
    at DestroyableTransform.afterTransform (/project/node_modules/readable-stream/lib/_stream_transform.js:93:3)
    at /project/node_modules/through2-concurrent/through2-concurrent.js:50:9
(node:99549) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:99549) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

From this line in concatLeftChunk()

        return Buffer.concat([runtime.csvLineBuffer, chunk]);

concatLeftChunk() is trying to pass this in:

[ <Buffer 41 6c>,
  '<long string from midway down the contents of my csv file>'

My setup is roughly like this:

  fs.createReadStream(file, {'encoding': 'utf-8'});
  .pipe(csv({},{flatKeys:true, objectMode: true}))
  .pipe(through2Concurrent.obj(
    {maxConcurrency},
    function (chunk, enc, callback) {
      let self = this;
      lineFunction(chunk).then(results => { self.push(results); callback() })
    }
  ))

It runs for a while, seemingly correct, and then errors.

Using this as shown above https://www.npmjs.com/package/through2-concurrent

[braco]

Removing objectMode from csv() and parsing the json further down seems to work:

  reader
  .pipe(csv({},{flatKeys:true}))
  .pipe(through2Concurrent.obj(
    {maxConcurrency},
    function (chunk, enc, callback) {
      let self = this;

      const _chunk = JSON.parse(chunk.toString());
      
      lineFunction(_chunk).then(results => { self.push(results); callback() })
    }
  ))

[furstenheim]

In the line of the source code if it is changed to:

 return Buffer.concat([runtime.csvLineBuffer, Buffer.from(chunk)]);

it also gets fixed