Streamstone
Streamstone copied to clipboard
yevhen
Duplicate detection entities redundant?
Hi, In your image of the schema you have an example of "Duplicate event detection is done by automatically creating additional entity for every event, with RowKey value set to a unique identifier of a source event".
I wanted to try to understand why this is necessary. Is is not enough that you have the event SS-SE-### entities and the SS-HEAD? Because you will get a conflict "by always including stream header entity with every write, making it impossible to append to a stream without first having a latest Etag". Additionally, you would get conflicts by trying to insert the same SS-SE-### entity twice as well. Could you help me understand why SS-UID is also needed? :)
UID is not the same as SeqID. You can use UID to implement idempotency, think prevention of duplicate form submission. Imagine, you're processing the command off the queue, it something fails between the moment you processed the command and deleting the message from the queue, the command will be retried and you re-processed it again. If you command has unique ID (GUID) you can use it as event UID.
Thanks. That is a great point. Could it be optional in case idempotency is not needed and you'd want to save some space (it adds up)?
What about deleting old UID entities, if I understand correctly there is no value in storing them?
Sure. I just tried using command id for event id, it seems that we have cases where one command raise multiple events and this does not work. :)
If we use EventInclude feature, included events are not considered as events by StreamStone, e.g. version is not incremented for those events and reading from stream do not return them.
What we have found and could work is that we can set the id on the first event while we left remaining events in a batch without id.
@mhertis for sure, they are just table entities. It's just a way how Streamstone exposes ETG. Nothing more