litefs icon indicating copy to clipboard operation
litefs copied to clipboard

LTX Event Data

Open benbjohnson opened this issue 2 years ago • 2 comments

The LTX file format is designed to store optional event data (similar to SSE) for each transaction file. This can be useful as replicas may need to know when and how data changes instead of polling their local copy for changes.

This will need to be implemented as something like a file handle so the application can write events to it. Notifications can also be implemented as a file handle or an HTTP endpoint.

benbjohnson avatar Jul 26 '22 14:07 benbjohnson

Is it possible to have something like : opening the -pos file and you get a stream of lines in the same format as the original pos file ? Multiple processes should be able to open the file and get all the lines.

Application side it would be just a matter of performing a while read_line(file) and read each line until we get the txid greater or equal to the one we need for consistency.

mildred avatar Aug 12 '23 13:08 mildred

It's probably possible but there can be weirdness between the size of a file during stat and then subsequent reads has a continuous stream of data. An HTTP endpoint or a unix socket is a better fit for this kinda thing.

benbjohnson avatar Aug 14 '23 00:08 benbjohnson