rust-rdkafka
rust-rdkafka copied to clipboard
`StreamConsumer` not waking
This StreamConsumer
racy behavior can be observed in the recent release, probably after the move to Event-based API.
Some of the events (stats, logs etc.) are processed internally and are consumed by the poll
returning None
. Since librdkafka
only wakes the queue when it transitions from empty -> non-empty, waker might be not called at any point in the future in this case.
Event order example:
- new message event
- new stats event
-
poll
processes stats and returnsNone
-
StreamConsumer
sets up waker and returnsPoll::Pending
- waker is never called by librdkafka since the queue never went empty
Probably connected issue: https://github.com/fede1024/rust-rdkafka/issues/638
I sketched out a potential solution to the problem: https://github.com/fede1024/rust-rdkafka/pull/666