node-pg-query-stream icon indicating copy to clipboard operation
node-pg-query-stream copied to clipboard

read and chunk large text column

Open poyaz opened this issue 6 years ago • 2 comments

hi, i create table with keys, body column and the body column have large text data and i wanna read all text value of body

Column Type Value
keys numeric[] array[1, 2]
body centered abcd... (20Mb)

now can split value of body to record OR read all value of body in chunk?

pg-query-stream just chunk record by record can chunk value of column?

poyaz avatar Aug 01 '18 20:08 poyaz

This looks like the same I am looking for.

I guess this project - node-pg-query-stream - is about streaming large amounts of rows without exhausting mem, but we also need a way to stream a large bytea column value from a single row.

Given the above example, I am thinking something in the lines of support for select body from t and then have the query result be a stream type in node for each row.body.

Is this already possible somehow?

polesen avatar Oct 09 '18 13:10 polesen

And likewise, we also need a way to stream data into a column when inserting. Currently, I am passing in a Buffer, but would be nice to be able to provide a stream, which was then streamed to the server.

polesen avatar Oct 09 '18 13:10 polesen