dice icon indicating copy to clipboard operation
dice copied to clipboard

Resume a QWATCH subscription

Open austince opened this issue 1 year ago • 2 comments

Perhaps I've missed this in the docs – is there a way to resume a QWATCH subscription from a point in the changelog stream (eg a bookmark in k8s watch, revision in etcd watch, offset in kafka, etc)?

EDIT: seems like I misunderstood QWATCH -- the entire snapshot appears to be sent on all changes. Is there any way to turn this into a snapshot-then-changelog (w/ inserts/retractions, eg) instead, as well as changelog-only? Redis Streams seems to tackle the changelog piece, but QWATCH is nice in that it can maintain the initial snapshot. Materialize's SUBSCRIBE offers nice options to customize this behavior: https://materialize.com/docs/sql/subscribe/#with-options

austince avatar Oct 02 '24 20:10 austince

QWATCH command output customization to support changelog generation is not on our roadmap for now. However, we would be interested in understanding your usecase to see if there are other ways in which dicedb could tackle them.

JyotinderSingh avatar Oct 17 '24 14:10 JyotinderSingh

Use case I'm thinking about is the k8s controller pattern (https://kubernetes.io/docs/concepts/architecture/controller/), where a Watch gets the full snapshot and then all changes afterwards, as well as being able to resume from a bookmark in the changelog instead of full snapshot.

austince avatar Nov 06 '24 17:11 austince