carp.core-kotlin icon indicating copy to clipboard operation
carp.core-kotlin copied to clipboard

Published 20 hours ago • verified publisher icon cph-cachet

Make sequence ID verification optional?

Open Whathecode opened this issue 3 years ago • 5 comments

Sequence IDs in data streams have a valid reason for being there:

  • detecting dropped data
  • allowing the backend to know no more data with older timestamps will be uploaded (unless sync points change, which is a separate topic)

However, to simplify client implementations, there has been a request to not have to be bothered with tracking sequence IDs. A specific infrastructure can therefore decide to omit sequence ID constraints. But, this inhibits using the current DataStreamSequence and DataStreamBatch objects, since these enforce these constraints.

Could there be an easy way for infrastructures to disable sequence ID constraints? Is this at all desirable?

Whathecode avatar Apr 01 '22 14:04 Whathecode

Could there be an easy way for infrastructures to disable sequence ID constraints? Is this at all desirable?

YES! This is highly desirable - this whole sequence ID issue is causing us many problems in CAWS right now.

bardram avatar Jun 20 '24 07:06 bardram

Other than redeployment needing the last IDs, are you running into other issues?

Whathecode avatar Jun 22 '24 13:06 Whathecode