decaton icon indicating copy to clipboard operation
decaton copied to clipboard

Isolate bursting key to prevent innocent keys are affected

Open ocadaruma opened this issue 1 year ago • 0 comments

  • In Kafka, it's common to use some entity's identifier (e.g. user-id) to ensure task processing ordering per entity.
  • At the same time, this tends to cause "hot-spot" key when there's an abusive task generation for single entity
  • When it happens, Decaton's processing thread could be occupied by such bursting key and could affect other innocent keys processing
  • It's nice to have a built-in feature to detect hot-spots and isolate workload to protect innocent keys

(Actually, this is a missing long-awaited Decaton feature that exists from early days...)

ocadaruma avatar Jul 26 '22 03:07 ocadaruma