python-logstash
python-logstash copied to clipboard
Try to reconnect if connection is lost (replace pika with kombu)
I need the AMQPLogstashHandler
to gracefully handle connection failure. Rather than implement this logic in the library as in #34, I recommend switching from pika to kombu.
In this PR I essentially just swapped the two libraries and made updates necessary to accommodate that change. This implementation uses kombu's producers pool for publishing, which handles connection failure gracefully. It might also be a good idea to add a max pool size option to the AMQPLogstashHandler
(see setting pool limits).
I also updated the socket logic to be more compatible with the SocketHandler
class. With the current implementation, we are made aware of connection problems because pika raises pika.exceptions.ConnectionClosed
, which escapes the except-block in SocketHandler.createSocket(). With kombu, OSErrors
are raised, which get silently swallowed, so I had to add a modified send()
.
@vklochan Hey! I saw you commented on the other PR and thought perhaps you had overlooked this one. Let me know if it addresses some of the issues with the previous PRs appropriately
I think this is a great addition to the library. Right now, messages a silently dropped after a connection failure