stream-snitch icon indicating copy to clipboard operation
stream-snitch copied to clipboard

Stream Snitch might not be reliable with smaller chunks.

Open tabarra opened this issue 5 years ago • 0 comments

It is not 100% guaranteed that stream-snitch will find a match even if the data you are looking for is present in the stream.
This happens specially if the length of the data is bigger than your chunks and when the buffer size is small.

Example:

const StreamSnitch = require('stream-snitch');
let sn = new StreamSnitch(
    /ab(\d{1,3})cd/g,
    (m) => {console.dir(m[1])},
    {bufferCap: 100}
);
sn.write('--------ab');
sn.write('123');
sn.write('cd');

In the case above, as soon as the 123 is written the buffer is flushed.

Why is this relevant?

When using piping a child process stdout, the chunk sizes can get very small (example), and although increasing buffer cap will reduce the frequency of dropped matches, it still occurs.

Possible fix:

Add a maxMatchSize to the options where the user can set the max size of the matching string (in the example below it would be 7).
When this option !== undefined the clearBuffer method would let the last maxMatchSize bytes when clearing the buffer.

StreamSnitch.prototype.clearBuffer = function() {
    if(typeof options.maxMatchSize !== 'undefined'){
        this._buffer = this._buffer.slice(-maxMatchSize);
    }else{
        this._buffer = '';
    }
  return this;
}

tabarra avatar Jun 12 '19 06:06 tabarra