ditto icon indicating copy to clipboard operation
ditto copied to clipboard

Outstanding responses for consumed commands in connections should throttle the consumption of messages from the source

Open thjaeckle opened this issue 2 years ago • 0 comments

During a session with @alstanchev we found out that for Ditto managed Kafka connections (probably same for other backpressuring connections) when

  • signals (e.g. "ModifyThing" commands) are processed
  • which require a response (responseRequired=true- which is also the default) to be send to a configured "replyTarget" address on the same Kafka
  • if those expected replies can not be delivered quickly enough
  • the consumption of new messages (from the connection "source") will not backpressure and consume slower from Kafka, but it will drop responses with warn message "Message is dropped as a result of backpressure strategy"

That was some kind of unexpected for us both - the connection should be well aware that it cannot deliver responses quickly enough and should slow down consumption of messages.

thjaeckle avatar Feb 07 '23 10:02 thjaeckle