faust
faust copied to clipboard
Consumers slowly die over time in the new Faust version 0.11.2
Steps to reproduce
- Everything was working fine in 0.11.1
- I upgraded to 0.11.2
- After a couple of received messages, the consumer stops receiving messages
Expected behavior
Consumer should continue receiving messages
Actual behavior
Consumer does not consume anymore, and slowly dies over time
Full traceback
[2024-08-09 10:17:48,341] [1] [ERROR] [^---AIOKafkaConsumerThread]: Stream stopped processing, or is slow for TP(topic='fn_forecast_init_metrics', partition=9) (last inbound 10.26 minutes ago).
There are multiple possible explanations for this:
1) The processing of a single event in the stream
is taking too long.
The timeout for this is defined by the stream_processing_timeout setting,
currently set to 300.0. If you expect the time
required to process an event, to be greater than this then please
increase the timeout.
2) The stream has stopped processing events for some reason.
3) The agent processing the stream is hanging (waiting for network, I/O or infinite loop).
Versions
- Docker Image: python:3.12.4
- Faust version: 0.11.2
- Kafka version: confluentinc/cp-server:7.6.0
Any particular reason why this is an issue particularly for v0.11.2? I know there have been similar issues for older versions of Faust.
see https://github.com/faust-streaming/faust/issues/175