channels_redis icon indicating copy to clipboard operation
channels_redis copied to clipboard

Add docs around using health_check_interval and tcp keepalive for pubsub layer?

Open justinttl opened this issue 10 months ago • 3 comments

When I was deploying pubsub layer using AWS Redis ElasticCache using default paramters, I noticed that occasionally my webservers will report

Connection closed by server.

during group_send PUBLISH from redis-py client returning b'' from a stale connection.

Adding health_check_interval and tcp keep alive seems to fix the problem

            "hosts": [
                dict(
                    host=...,
                    port=...,
                    socket_keepalive=True,
                    health_check_interval=15,
                    retry_on_timeout=True,
                )
            ],

I am not sure why this issue is not present with the default implementation. (I wonder if it is because the "polling" from default impl keeps the connections in the pool alive). For PubSub, I think it makes sense since the connection could get stale if there are no activity for a while?

Anyhow, wondering if it is worth adding a section to the README regarding these config values for pubsub.

cc @acu192 as you have a lot of experience running this in production.

justinttl avatar Jan 21 '25 16:01 justinttl

@justinttl No problem with adding a note about this to the README.

carltongibson avatar Jan 23 '25 10:01 carltongibson

@justinttl would you be willing to write up the PR?

bigfootjon avatar Jan 25 '25 20:01 bigfootjon

Sure - should be free later this week to throw the changes together.

justinttl avatar Jan 27 '25 16:01 justinttl