quickwit icon indicating copy to clipboard operation
quickwit copied to clipboard

Kafka source crashes when experiencing backpressure from indexer and waits for too long

Open guilload opened this issue 3 years ago • 1 comments

"Maximum application poll interval (max.poll.interval.ms) exceeded".

guilload avatar Oct 07 '22 11:10 guilload

Reported in #2080.

cc @guidao

guilload avatar Oct 07 '22 11:10 guilload

The Kafka source now does a better job at logging the values (max poll interval and session timeout) that can lead to the consumer crashing when experiencing backpressure.

Downstream, the indexing pipeline has been improved to lead to less backpressure.

guilload avatar Nov 15 '22 21:11 guilload