kafka-sink-azure-kusto icon indicating copy to clipboard operation
kafka-sink-azure-kusto copied to clipboard

Adding null check to handle rare NPE case

Open preston-m-price opened this issue 2 years ago • 0 comments

Pull Request Description

In our deployment we rarely hit an NPE here, which typically results in duplicates due to the unhandled exception causing a premature exit from the WorkerSinkTask.commitOffsets method. In this failure mode we've already published batches to kusto, but we don't update the offsets due to an exception here.

I have not yet fully understood how we get into this state, where a TopicPartitionWriter doesn't exist for a TopicPartition.

Future Release Comment

Fixed the source of a NullPointerException that could result in duplicate records sent to Kusto.

Breaking Changes:

  • None

Features:

  • None

Fixes:

  • None

preston-m-price avatar Nov 01 '22 18:11 preston-m-price