Piece-wise full table scan

[prataprc]

Rdms indexes can support full table scan, by providing an iterator handle to iterate over every single entry in sort order. Since by design rdms also provide concurrent write operation, while the scan is in progress, this may lead to unstable scan if not implemented properly.

One mechanism used by Rdms to implement stable scan is piece-wise scan. Again this mechanism is applicable only when entries are indexed in lsm mode. The basic idea is to do range scan in short batches, say 1000 entries for each batch, and drop the read-handler after the batch is read. There by, avoiding resource pressure on index and/or holding its rw-locks. To begin with caller should compute the seq.no. range (start, end] that should be picked from the index and a begin-key:

• start-seqno means all entries and versions whose seqno is before the start-seqno is already seen by the caller.
• end-seqno is the index-sequence-number just before starting the full-table-scan.
• begin-key, for every batch, would be the last batch's last entry-key. It is implied that the first batch's begin-key would be Bound::Unbounded.
• Only entries and versions whose seqno is between (start, end] sequence number are returned.
• Only entries that come after the begin-key are returned.
• Once the iteration for a batch exceeds a limit the iterator is dropped and new batch is started.