zstd icon indicating copy to clipboard operation
zstd copied to clipboard

CLI flag to continuously watch compressed file for changes

Open tdebock-drw opened this issue 6 months ago • 7 comments

Is your feature request related to a problem? Please describe. When working in an environment that uses zstd to compress files as they are being written (in my case large logs), having a way to display the content of a compressed log file as it's being written to (updating as data is appended to the file), would be very convenient. This is however difficult, since zstdless just pipes the output of zstdcat (which obviously does not provide a constant stream of data as the file updates) into less. There is currently no way that I know of to achieve this desired behaviour.

Describe the solution you'd like I forked zstd and changed it to compile an additional (slightly modified zstdcat) binary that continuously reads, decompresses, and outputs newly appended data as it's being appended to the compressed input file, not exiting when it reaches EOF. Using this, I slightly modified the zstdless script to pipe this continuous stream of data into less, making the behaviour of zstdless much more similar to normal less. I would like to see this behaviour implemented in zstd, by the addition of a CLI flag to be passed to zstd when using it for decompression, allowing for an improved implementation of zstdless. If deemed fitting for the project, I could modify my solution to be incorporated into the project in this way.

tdebock-drw avatar Jul 03 '25 15:07 tdebock-drw