Subscriptions don't seem to reconnect if Redis dies

[jwoertink]

In Cable-cr we have this instance setup

https://github.com/cable-cr/cable/blob/a8f7f3d9d8218078ab270720738dc33e0858775b/src/cable/server.cr#L35

it calls subscribe when the server is booted, and ran within a spawn

https://github.com/cable-cr/cable/blob/a8f7f3d9d8218078ab270720738dc33e0858775b/src/cable/server.cr#L152-L154

If the redis server gets rebooted, then that connection will throw a Redis::CannotConnectError, and never tries to reconnect. Currently the only way to resolve this within our app seems to be to just reboot the entire app and tell users to refresh their browsers so the websockets will all reconnect and such.

Here's a local stacktrace we see when testing this by booting our app, restarting redis, then trying to send a chat message.

web          | Unhandled exception in spawn(name: Cable::Server - subscribe): Socket::ConnectError: Error connecting to 'localhost:6379': Connection refused (Redis::CannotConnectError)
web          |   from lib/redis/src/redis/socket_wrapper.cr:12:5 in 'initialize'
web          |   from lib/redis/src/redis/connection.cr:8:3 in 'new'
web          |   from lib/redis/src/redis.cr:157:19 in 'connect'
web          |   from lib/redis/src/redis.cr:147:7 in 'ensure_connection'
web          |   from lib/redis/src/redis.cr:133:5 in 'strategy'
web          |   from lib/redis/src/redis.cr:311:7 in 'command'
web          |   from lib/cable/src/cable/monkeypatch/redis.cr:8:9 in 'void_command'
web          |   from lib/redis/src/redis/commands.cr:1741:7 in 'subscribe'
web          |   from lib/redis/src/redis/commands.cr:1746:7 in 'subscribe'
web          |   from lib/redis/src/redis/commands.cr:1731:7 in 'subscribe'
web          |   from lib/cable/src/cable/server.cr:153:9 in '->'
web          |   from /home/dev/.asdf/installs/crystal/1.4.0/share/crystal/src/fiber.cr:146:11 in 'run'
web          |   from /home/dev/.asdf/installs/crystal/1.4.0/share/crystal/src/fiber.cr:98:34 in '->'
web          |   from ???

[jwoertink]

Maybe related: https://github.com/ysbaddaden/pool/issues/2

[kostya]

i think this problem only with subscription, because it is special command, any other commands reconnects ok. This is not related to pool at all.

[kostya]

fixed in master