node-csv-stream
node-csv-stream copied to clipboard
Pause maybe not working as intended
My understanding of pause() with respect to streaming a csv would be to pause the stream at the current record. This would allow async processing to occur before resuming the stream to get the next record.
What's currently happening is the buffer is fully processed before _paused is taken into account.
var count = 0;
var csvStream = new csv.createStream({
endLine : '\n',
escapeChar : '"',
enclosedChar: '"'
})
var readStream = fs.createReadStream(targetPath)
csvStream.on('error', function(err){
console.log(err)
})
csvStream.on('data', function(data){
count++
console.log('pre pause', count)
csvStream.pause()
//I would normally have async code here with a csvStream.resume() in a callback/promise
})
csvStream.on('end', function(){
console.log("we're done")
//cb()
})
readStream.pipe(csvStream)
I would expect the stream to pause at each record, but it goes through until the end of file (or buffer in this case) before actually entering the paused state.
Is the intended behaviour to process the entire file/buffer before allowing pause() to occur?
Yes that's the expected behavior but I admit that's not really usable for your use case. You can submit a PR with a fix I will merge it.
+1 . why it is not behaving as intended? I can swear it was working for me 1 week ago :o
@scopsy that method does not working for me either, how you do the working method btw ?