nakadi
nakadi copied to clipboard
Queue publishing of events for the same key in a batch
When hash partitioning (and/or log compaction) is used this approach ensures that the order of events is preserved for each key in the face of intermittent publishing errors.
For now we simply mark any events that were not attempted for publishing as aborted. Later this can be improved to implement retry of failed events.
No change of behavior when no event keys are set (random or user-defined partitioning): the whole batch is submitted as a single chunk.