python-logstash icon indicating copy to clipboard operation
python-logstash copied to clipboard

Try to reconnect if connection is lost (replace pika with kombu)

Open matt-snider opened this issue 8 years ago • 2 comments

I need the AMQPLogstashHandler to gracefully handle connection failure. Rather than implement this logic in the library as in #34, I recommend switching from pika to kombu.

In this PR I essentially just swapped the two libraries and made updates necessary to accommodate that change. This implementation uses kombu's producers pool for publishing, which handles connection failure gracefully. It might also be a good idea to add a max pool size option to the AMQPLogstashHandler (see setting pool limits).

I also updated the socket logic to be more compatible with the SocketHandler class. With the current implementation, we are made aware of connection problems because pika raises pika.exceptions.ConnectionClosed, which escapes the except-block in SocketHandler.createSocket(). With kombu, OSErrors are raised, which get silently swallowed, so I had to add a modified send().

matt-snider avatar Aug 11 '16 20:08 matt-snider

@vklochan Hey! I saw you commented on the other PR and thought perhaps you had overlooked this one. Let me know if it addresses some of the issues with the previous PRs appropriately

matt-snider avatar Aug 17 '16 14:08 matt-snider

I think this is a great addition to the library. Right now, messages a silently dropped after a connection failure

cp2587 avatar Oct 24 '16 07:10 cp2587