noria icon indicating copy to clipboard operation
noria copied to clipboard

Allow trading off consistency for performance

Open jonhoo opened this issue 5 years ago • 0 comments

Currently, we "swap" the Reader after every batch we process. This ensures that views are always as fresh as we can make them (subject to write propagation delays), but also means we incur the cost of waiting for readers to exit the current map for every batch. Users would likely see significant speedups if we amortized the cost of swaps by doing them less often, though this would come at the cost of an increase in the delay before writes become visible.

This should probably be an option that is exposed to users, with sufficient warning labels about "eventualness". Care must also be taken to ensure that we eventually swap the maps, even if there are no more writes, so that reads don't remain eternally stale.

jonhoo avatar Sep 19 '18 16:09 jonhoo