core icon indicating copy to clipboard operation
core copied to clipboard

recoverylog: track multiple segments per file

Open jgraettinger opened this issue 5 years ago • 0 comments

As an operator, I'd like recovery logs to track multiple segments of written journal range for every file, up to a bound. For RocksDB workloads, all writes of SST files co-occur closely in the log, but writes to the MANIFEST may be interspersed with many other files. If the FSM machinery tracked disjoint segments of the most recent MANIFEST, it allows playback to skip through portions of the log it's unable to skip today, improving recovery times.

The FSM cannot track every segment of every file, so there must be an upper bound, and the FSM must intelligently collapse Segments to not violate that bound (eg, by picking Segments to merge which least-impact readers).

jgraettinger avatar Nov 18 '19 16:11 jgraettinger