node-filewalker icon indicating copy to clipboard operation
node-filewalker copied to clipboard

[question] How to configure the number of bytes read?

Open thiagodp opened this issue 7 years ago • 3 comments

I would like to increase the number of bytes read each time. This is especially important when dealing with large files. How can I do this with node-filewalker? Thanks

thiagodp avatar Dec 18 '17 17:12 thiagodp

Hi, I just discovered how to do the trick:

    filewalker( path, options )
        .on( 'stream', function( rs, p, s, fullPath) {
           // Adjust the number of bytes read each time
            rs._readableState.highWaterMark = 1024 * 1024; // 1 MB
            ...
        } )
        ...

However, it would be nice whether filewalker could include such property in its options.

Maybe to include all the ReadStream options, in a property like readStream, to be used when the ReadStream is being created.

let readStreamOptions: {
      highWaterMark: 1024 * 1024 // 1 MB
   };
filewalker( path, options, readStreamOptions );

This would also allow do things like that:

// will read simple.txt from byte 90 to 99
filewalker( '.', { matchRegExp: /simple\.txt/ }, { start: 90, end: 99 } );

thiagodp avatar Dec 18 '17 17:12 thiagodp

Pull request added.

thiagodp avatar Dec 18 '17 18:12 thiagodp

I added the PR 19, which allows to pass readStream with options:

var options = {
  readStream: {
    highWaterMark: 1024 * 1024 // 1 MB
  }
};
filewalker( path, options );

thiagodp avatar Dec 21 '17 03:12 thiagodp