node-csv-stream icon indicating copy to clipboard operation
node-csv-stream copied to clipboard

Pause maybe not working as intended

Open davebriand opened this issue 10 years ago • 3 comments

My understanding of pause() with respect to streaming a csv would be to pause the stream at the current record. This would allow async processing to occur before resuming the stream to get the next record.

What's currently happening is the buffer is fully processed before _paused is taken into account.

    var count = 0;
    var csvStream = new csv.createStream({
        endLine : '\n',
        escapeChar : '"',
        enclosedChar: '"'
    })
    var readStream = fs.createReadStream(targetPath)
    csvStream.on('error', function(err){
        console.log(err)
    })
    csvStream.on('data', function(data){
            count++
            console.log('pre pause', count)
            csvStream.pause()
            //I would normally have async code here with a csvStream.resume() in a callback/promise 
        })
    csvStream.on('end', function(){
            console.log("we're done")
            //cb()
        })

    readStream.pipe(csvStream)

I would expect the stream to pause at each record, but it goes through until the end of file (or buffer in this case) before actually entering the paused state.

Is the intended behaviour to process the entire file/buffer before allowing pause() to occur?

davebriand avatar Oct 04 '15 11:10 davebriand

Yes that's the expected behavior but I admit that's not really usable for your use case. You can submit a PR with a fix I will merge it.

lbdremy avatar Oct 04 '15 16:10 lbdremy

+1 . why it is not behaving as intended? I can swear it was working for me 1 week ago :o

scopsy avatar Jan 25 '16 16:01 scopsy

@scopsy that method does not working for me either, how you do the working method btw ?

slaveofcode avatar Nov 01 '17 07:11 slaveofcode