silverback icon indicating copy to clipboard operation
silverback copied to clipboard

Support for master data sequences

Open BEagle1984 opened this issue 3 years ago • 0 comments

Support a new type of sequence to deal with master data replication. Some headers will be used to delimit the dataset.

It should ideally be possible to publish a collection of messages and have the Silverback producer automatically add the necessary headers. The consumer should recognize the dataset and create an ISequence to deal with it as a unit of work, where the subscriber recognizes when all messages have been received (subscribing to the IMessageStreamEnumerable, that will complete when the entire set has been consumed).

The headers should be as follows.

Header Description
x-dataset-last Marks the last dataset element in this partition.
x-dataset-first-offset Starting offset of the dataset in this partition. Present in all records except the first one of each partition.
x-dataset-version Optional. Monotonic increasing number identifying the dataset version.
x-dataset-index Sequence number, within a version. Starts with zero.
x-dataset-count Optional. Total number of records.

BEagle1984 avatar Jan 11 '21 17:01 BEagle1984