recreate bindings on failures

[julianhille]

Issue

Binding to an exchange should happen on every failure in the listening loop. I've seen several times (not sure what exactly triggers it), that if rabbitmq goes down and comes up again, the bindings are lost. If the "while True listen loop" then still runs it will never listen again to the queue again. This is due to the fact that the rabbitmq exchange binding / channel creatoin happens before the while loop.

This ends up in something similar to a split brain. The machine is able to send events to a room but never receives any.

Why not, on specific errors or every time, recreate the binding to the exchange?

[miguelgrinberg]

Is this something you are interested in contributing? I use Redis myself for my projects, and have improved and made it more robust and fault tolerant over time. We want a similar effort for the Kombu and aiopika classes, which I have less experience on.

[julianhille]

i'm actually not sure if its correct what i'm saying.

i currently try things like:

• disconnecting the ip
• deleting the channel

will try to gather more info, until some minutes ago i thought one of them it is.

currently looking at:

• deleting the exchange

This seems as it would not recover.

BUT this does not reflect what im seeing. im seeing:

• a gunicorn running
• an api / namespace answering to events
• no exceptions in logs
• ip is not connected at all anymore (none of the two connections i should see)
• binding is missing
• channel is missing
• restart of the container fixes it

maybe you have any idea what to test.