asyncio redis cluster: fixed reconnection when whole cluster goes down

[adaamz]

Pull Request check-list

• [x] Do tests and lints pass with this change?
• [x] Do the CI tests pass with this change (enable it first in your forked repo and wait for the github action build to finish)?
• [X] Is the new or changed code fully tested?
• [ ] Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?
• [ ] Is there an example added to the examples folder (if applicable)?
• [ ] Was the change added to CHANGES file?

Description of change

When redis cluster goes completely down (all nodes are offline) then redis-py library is unable to reconnect to them when at least startup nodes are back online. This workaround worked for me, but I'm not sure if there is some more efficient way to achieve same bugfix.

I testesd it isolately with spawning minimal redis cluster in docker and then taking it down and up after few seconds to test how the app reacts on this. My python test script looks like this:

import asyncio
import time
import traceback

from redis.asyncio.cluster import ClusterNode, RedisCluster

redis_host = "172.26.0.2"
redis_port = 6379
key = "some_key"


def prepare_redis_async():
    return RedisCluster(
        ssl=False,
        startup_nodes=[
            ClusterNode(host=redis_host, port=redis_port)
        ],
        socket_connect_timeout=3,
        socket_timeout=60,
        require_full_coverage=True,
        max_connections=5,
        cluster_error_retry_attempts=20
    )


redis_client_async = prepare_redis_async()


async def async_get_value():
    x = await redis_client_async.get(key)
    print(x)
    await redis_client_async.set(key, "async_test")

if __name__ == '__main__':
    loop = asyncio.get_event_loop()

    while True:
        try:
            loop.run_until_complete(async_get_value())
        except:
            print(traceback.format_exc())

        time.sleep(0.1)