lento icon indicating copy to clipboard operation
lento copied to clipboard

createRowStream is suboptimal

Open vweevers opened this issue 7 years ago • 0 comments

Because it sets highWaterMark to 0 (though documented as 16):

https://github.com/vweevers/lento/blob/8670652f34e5c2f98145b669ff269efa81ba4c5f/lib/client.js#L176-L180

This makes sense for a page stream, because it means it won't make preemptive HTTP requests. But for a row stream it means it'll push only one row (or "chunk") at a time, possibly hurting throughput:

https://github.com/vweevers/lento/blob/8670652f34e5c2f98145b669ff269efa81ba4c5f/lib/query-stream.js#L68-L76

I.e. the while loop above never loops. It might be easier to push everything at once (ignoring the return value of push). It's already occupying memory anyway.

Also, when the next readable-stream is out (with https://github.com/nodejs/node/commit/1e0f3315c77033ef0e01bb37c3d41c8e1d65e686), I can maybe get rid of:

https://github.com/vweevers/lento/blob/8670652f34e5c2f98145b669ff269efa81ba4c5f/lib/query-stream.js#L46-L48

For now, I recommend to use createPageStream instead.

vweevers avatar Jun 06 '18 13:06 vweevers