fst
fst copied to clipboard
Optional connection argument to compress_fst
Writing directly to a connection could help conserve memory. Suggested at https://github.com/richfitz/storr/pull/111#issuecomment-509136783.
Hi @wlandau, great, thanks for submitting!
We could go a little further and adapt compress_fst() to:
compress_fst(con_in, con_out = NULL, compressor = "ZSTD", compression = 0, hash = FALSE)
So, it would also accept a connection as the input (con_in) and/or the output (con_out).
That makes the function much more flexible. For example, with a connection selected for the input and for the output, a file can be compressed with very little RAM overhead.
When a raw vector is selected for the input and a connection for the output, we get to your scenario, where the compressed stream is directly written to disk...
Nice! An optional output connection would certainly enhance versatility.