equinox
equinox copied to clipboard
Cosmos: Provide ability to separate storage of snapshots
The unfolds mechanism, as documented in #50 provides the ability to efficiently stash relevant data facilitating reading of state based on a point read.
However, this is can be suboptimal in the cases of larger snapshot states:-
- the data gets compressed and travels each time
- writing/updating occasions read and write costs within the sync stored procedure each time
- competing writers pay all but the update case in the case of a re-run of the same transaction
- readers not interested in/unable to process a specific snapshot schema still have to pay the price to transport it
- each update goes to the changefeed triggering more read costs
- the size and cost of the snapshots is harder to separate out
- snapshots contribute to partition size limits
Thus it makes sense to do some, or all of the following in the case where there is a projector in play and there is an aux collection (in the case of multiple writers one might even consider making an aux collection even if you don't have a projector):
- maintain the snapshots in the
auxcollection, where one is not paying for changefeed re-reads of the same data - reduce the write frequency for snapshots (when the write happens should be deterministic for competing writers)
- implement ability for stored proc to maintain events since snapshot in tip when not updating snapshot
- write to a nominated collection (typically aux), with a deterministic guid based on the case / event type tag of the union as a point write
- refer to the snapshot in the unfolds; automate loading of the referenced snapshot if picked by
isOrigin - expose the ability to customize the snapshot loading process (but batteries should be included)
- (perhaps do a sample showing parallel reads of the Snapshots and the Tip?)
The designs for #145 and #157 should consider the potential overlap with this requirement.