client_async: Allow throwing an exception upon socket error during wakeup
Motivation
When wakeup() is called, we sometime notice that we get an endless prints: "Unable to send to wakeup socket!". Those prints are spamming the logs.
Description
This commit aims to address it by allowing restating the application via an intentional exception raise. This behavior is configurable and its default is backward compatible.
We see this issue with kafka-python 2.0.2
@dpkp Can u please review this PR?
@shimon-armis do you have a traceback of the logs in question, and some sample code?
@shimon-armis do you have a traceback of the logs in question, and some sample code?
Hey @wbarnha, It's been a while :) but here are some snippet I found in my slack's DM.
Note, It isn't a complete traceback.
W0221 08:13:16.082472 kafka.client 1-140694973044480 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:331] Node 11 connection failed -- refreshing metadata W0221 08:13:16.083114 kafka.client 1-140694973044480 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:331] Node 12 connection failed -- refreshing metadata W0221 08:13:16.084225 kafka.client 1-140694973044480 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:331] Node 13 connection failed -- refreshing metadata W0221 08:13:16.224422 kafka.client 1-140694981437184 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:331] Node 19 connection failed -- refreshing metadata W0221 08:13:16.254804 kafka.client 1-140694589003520 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:331] Node 20 connection failed -- refreshing metadata W0221 08:13:16.324988 kafka.client 1-140694981437184 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:937] Unable to send to wakeup socket! W0221 08:13:16.425574 kafka.client 1-140694981437184 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:937] Unable to send to wakeup socket! W0221 08:13:16.526113 kafka.client 1-140694981437184 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:937] Unable to send to wakeup socket! W0221 08:13:16.626606 kafka.client 1-140694981437184 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:937] Unable to send to wakeup socket! W0221 08:13:16.727129 kafka.client 1-140694981437184 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:937] Unable to send to wakeup socket! W0221 08:13:16.827740 kafka.client 1-140694981437184 /home/runner/pipeline.runfiles/pip_kafka_python/site-packages/kafka/client_async.py:937] Unable to send to wakeup socket!
@wbarnha Note that the suggested change is an acceptable practice, under the circumstances. I probably missed your previous comment but will happy if we can get this done quickly 🙏
I'm resuming work on this in https://github.com/wbarnha/kafka-python-ng/pull/134!