Advanced Streams

[thedodd]

In GitLab by @doddzilla on Oct 17, 2019, 10:17

possible designs

unique id

• a stream which just checks an ID value before the entry is submitted.
• if the ID is unique, the entry will be committed, else it will be rejected with a unique ID violation.

use cases

• Exactly-once semantics enforced on the server. The basic stream's transactional ack+pub provides the same guarantees for normal stream processing patterns.
• Guard against duplicate entries generally speaking.

tagged streams

• this supersedes unique ID streams.
• stream entries can be published with a set of associated tags.
  • the tags are declared adjacent to the payload, not as part of the payload itself. It is metadata.
  • the tags may be declared in two buckets:
    • constrain: the given tags will be added to the stream's index to guarantee uniqueness.
    • remove: the given tags will be removed from the stream's index.
  • compound indices / constraints can be declared by using a standardized tagging convention.
  • each tag published in the constrain bucket will be checked for uniqueness, and an error will be returned if the values are not unique.
  • all tag fields & values must be string values.