restate icon indicating copy to clipboard operation
restate copied to clipboard

Re-consider the kafka ingress ordered delivery semantics

Open slinkydeveloper opened this issue 1 year ago • 0 comments

In our kafka implementation we inherited much of the constraints about ordering from the delivery guarantees of the keyed/unkeyed service.

This is not needed anymore, and we should implement the following new semantics now:

  • For Virtual objects with exclusive handlers, the kafka record key MUST be UTF-8 valid, and you get ordering on key basis.
  • We forbid subscribing to Kafka Virtual objects with shared handlers. This is unclear how it should work, and what are the user expectations. Once it's more clear, we can allow it.
  • For Services, we wanna support two delivery semantics:
    • Max fanout no ordering.
    • Ordering on topic/partition basis. For this we can profit from the recent changes to ids and service status/invocation status table split, to define on an invocation basis whether we wanna have ordering and on which key.

slinkydeveloper avatar Apr 17 '24 17:04 slinkydeveloper