Consumers start hanging with AIOKafkaConnection._read() Cancelled Error

[vineetgoel]

The consumers start hanging at some point with the following error being logged:

[2017-11-07 18:17:47,547: ERROR] Task exception was never retrieved
future: <Task finished coro=<AIOKafkaConnection._read() done, defined at /home/foo/env/lib/python3.6/site-packages/aiokafka/conn.py:194> exception=CancelledError()>

What might be causing the error? Should we be crashing the consumer upon getting the error?

This is with aiokafka==0.3.1

[vineetgoel]

I confirmed that this isn't making the consumer hang and is not really having any adverse effects. However, we should be handling these exceptions better.

[vineetgoel]

Any updates on this?

I repeatedly keep getting these errors, don't think it is fatal though.

[tvoinarovskyi]

Could you describe how you use the consumer? The error seems strange, it's like you close the consumer the same moment it connects... It is probably not critical, but I still fail to understand when it fails.

[vineetgoel]

I am not closing the consumer. This happens while the consumer is idle for sometime waiting for messages on certain partitions.

[tvoinarovskyi]

Ohh, this is probably a fail on idle connection closure. The connection is closed if not active for some time, probably this log is a side effect there. Will investigate, thank

[vsel]

I have the same problem. After 10 minutes of idle connection.

[tvoinarovskyi]

Ignore it, it's a bug, but should have no impact on operability

[vsel]

OK. Thank you. I was confused because I just produce data and read nothing.

[tvoinarovskyi]

@vsel Sorry, but I still have problems reproducing this. Could you provide your Producer configuration and if possible the script, that can reproduce the issue.

[tvoinarovskyi]

Ok, this is not the idle thing... The connections_max_idle_ms config did not work at all =)

[vsel]

Indeed! Do you still need my configuration for reproducing the issue? P.S. Sorry for delay.

[tvoinarovskyi]

Yes, please. Want to track it down

[vsel]

I can’t reproduce this bug with virtual kafka environment, so I’ll try to chat about this issue with devOps colleagues. I’m sorry. May be I can check something on production environment?

[tvoinarovskyi]

This should be fixed as of 0.4.0, but will leave open until someone can confirm it.

[bt-wil]

I haven't ran into this issue using aiokafka==0.4.0

[myaspm]

I'm still getting this errors with version 0.4.0. Don't think that it impacts anything but still..