CSV File Line Hook how to tell to ignore or skip based on return value?

[travis5491811]

CSV File Line Hook how to tell to ignore or skip based on return value?

I'm parsing a CSV to json and i want to ignore lines based on a conditions, below is an example but the conditions could be anything:

  readStream.pipe(csvtojson()
    .preFileLine((fileLineString, lineIdx)=>{
			return new Promise((resolve2,reject2)=>{
				// async function processing the data.
				if (lineIdx === 2){
					// return fileLineString.replace('some value','another value')
					console.log(fileLineString);
					resolve2(fileLineString);
				}
        resolve2(null);
			});      
	  })
    
  
	).pipe(
		writeStream.on('finish', function () {
			console.log(`Finished writing ${originalFileName}.csv to ${originalFileName}.json`);
		}).on('close', () => {
			console.log(`${originalFileName}.json file closed and ready for reading.`);
			let end= Date.now();
			let timeSpent=(end-begin)/1000+'secs';
			totalTime += (end-begin);
			console.log('Convert CSV To JSON Duration', timeSpent);
			resolve(true);
		}).on('error', reject)
	);

What i'm trying to make happen is the json output to skip or not process when i reject or maybe resolve(null). Instead, its creating row like below for each of the resolve(null)

{"null":"null"}
{"x":"2","field2":"asdfasfsdfs"}
{"null":"null"}
{"null":"null"}
{"null":"null"}
{"null":"null"}

And it's also making the first row of what i ant to keep have first key of 'null' and replaces all the column names with field1, field2, etc

How do i get it to not write anything (skip rows) based on a condition?

[travis5491811]

I guess preFileLine is not for this but didn't get far with subscribe either

  .subscribe((jsonObj)=>{
    // asynchronousely
    return new Promise((resolve,reject)=>{
        if (jsonObj['!FIRSTNAME'] === 'Ryan') {
          jsonObj['!FIRSTNAME'] = 'SAM123';
          resolve();
        } else {
          jsonObj = {};
          resolve();
        }
    })
})

In the above example, it does change the value to "SAM123" for the matching records, but in the else statement i set the obj to empty and it still writes the full object for all other lines instead of not writing anything.

[travis5491811]

The following works to at least set the other fields to empty objects.

		.subscribe((jsonObj)=>{
			// asynchronousely
			return new Promise((resolve2,reject)=>{
				if (jsonObj['name5'] === 'af8c6') {
					jsonObj['name5'] = 'asdfasfsdfs';
				} else {
					// jsonObj = {};
					for (var key in jsonObj) delete jsonObj[key];
				}
				resolve();
			});
		})

I'm not sure why jsonObj = {} does not set to empty object on output. Using this as alternative for (var key in jsonObj) delete jsonObj[key]

Now i get the following:

{}
{"x":"2","name5":"asdfasfsdfs"}
{}
{}
{}
{}

But still trying to get just the rows i want

{"x":"2","name5":"asdfasfsdfs"}

[silesky]

used the preRawData api.

 csv()
    .fromFile(path)
    .preRawData(str => {
      const lines = str.split('\n');
      const trimmedStr = lines.filter(line => /\w/.test(line)).join('\n');
      return trimmedStr;
    });

[AlexandrePalo]

used the preRawData api.

 csv()
    .fromFile(path)
    .preRawData(str => {
      const lines = str.split('\n');
      const trimmedStr = lines.filter(line => /\w/.test(line)).join('\n');
      return trimmedStr;
    });

According to my tests, for very large lines the raw data is split in chunks that doesn't always correspond to lines. Thus, it's not possible to filter on a specific column value.

However, preFileLine's hook is working fine. Skip first line for headers and return empty string for filtered lines.

[vishnupeas]

Why is this issue not being addressed, is this something pointless to do? I have tried "csv" npm package too. It also had no option to address the skipping of records by giving a certain condition.