kafka-python
kafka-python copied to clipboard
dpkp
Propagate future.exception in producer/kafka.py _wait_on_metadata
Propagate any exceptions that happened during the future; otherwise _wait_on_metadata will time out even though the error is being continuously reflected in the metadata status fetched from the broker. Issue #2078
This change is 
Wouldn't this prevent retrying exceptions that are likely to be resolved within the timeout? Are there specific exceptions that you are concerned about here?
Any permission issues are not transient. I was testing with locked down topics and even though the broker sent the error back the library ignored it. Also, any attempt to connect with a misconfigured authentication scheme (ssl, sasl, whatever) will also never correct itself.
It’s been a bit since I looked at it, but propagating the exception solved at least two issues I encountered during testing. Presumably the same issues could exist on the consumer side but I was only testing the producer side.
I guess if you really felt like it you could split the exceptions into transient and permanent exceptions and only propagate the permanent ones, but it seems like that would be most of them.
Sent from my iPad
On Sep 7, 2020, at 7:25 PM, Dana Powers [email protected] wrote:
Wouldn't this prevent retrying exceptions that are likely to be resolved within the timeout? Are there specific exceptions that you are concerned about here?
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.
It looks like the java client only raises on invalid or unauthorized topics, or on a connection authentication failure. Other failures are not raised and will get retried subject to the timeout.