line-reader icon indicating copy to clipboard operation
line-reader copied to clipboard

Does it save the whole file to RAM?

Open rfonovicell opened this issue 3 years ago • 1 comments

Sorry if my question is dumb, but I believe the whole point of reading a file line-by-line is not having to load it entirely into RAM in order to read it. So if I have, for example, an 100 MB file, will it be loaded into RAM before the code can read it?

rfonovicell avatar Jan 28 '22 10:01 rfonovicell

It uses a read stream internally, and additionally buffers one line at a time using a separator character, if I'm reading the source correctly. So only one line is buffered into memory at a time.

What makes streams unique, is that instead of a program reading a file into memory all at once like in the traditional way, streams read chunks of data piece by piece, processing its content without keeping it all in memory.

src: https://nodesource.com/blog/understanding-streams-in-nodejs/

ref: https://nodejs.org/api/fs.html#filehandlecreatereadstreamoptions

ungoldman avatar Jul 12 '22 18:07 ungoldman