`amqp-kafka` feature

[jfallows]

AMQP 1.0 specification

• binding-amqphandles network protocol
  • [ ] encoding and decoding amqp protocol frames
  • [ ] lifecycle of sender and receiver application level streams
  • [ ] flow control of multiple receiver application streams on the same network stream
  • [ ] security guard integration via SASL frames, expiration and challenge-response via internal link name
• binding-amqp-kafka handles the mapping
  • [ ] link stealing, same client container
  • [ ] attempt to align delivery state with kafka topic progress
    • limited to 32 octets, use server side state to dereference larger states as needed
    • session state storage abstraction
      • log-compacted kafka topic, queryable by clientId message key, configurable name
      • subscription state as log-compacted topic, queryable by clientId message key, configurable name
      • support out-of-band attach or detach for client
      • ideally state storage abstract can support different implementations; memory, hazelcast, redis, ...
  • [ ] sender settle mode settled as produce, acks none
  • [ ] sender settle mode unsettled as produce, acks in_sync_replicas
  • [ ] sender capability amqp:local-transactions as produce, acks in_sync_replicas and transactional producerId
  • [ ] receiver settle mode unsettled as stateful fetch with consumer id for message delivery retry
  • [ ] queue receivers use tiered consumer group members
    • system consumer group member reads messages from topic partition
    • system consumer group members redistributed across zilla instances
    • virtual consumer group members read messages with same key to support consistent ordering per key
    • virtual consumer group members redistributed within same zilla instance
    • detach system consumer group member instance when no remaining virtual members to avoid delivery starvation
  • [ ] message filters
    • no-local as not-equal sender client-id kafka message header
    • message selector as kafka message headers filter
  • [ ] support load balancer bypass via server redirection