Async Cluster does not attempt reconnect when max connections is reached

[stinovlas]

Version: 5.0.1

Platform: Python 3.11.6 on Debian trixie

Description:

redis.asyncio.RedisCluster seems not to handle reconnections correctly, when maximum number of connections is reached. It immediately raises MaxConnectionsError without any reconnection attempts.

Minimal working example

import asyncio
from redis.asyncio import RedisCluster

async def subtest(redis: RedisCluster):
    await redis.set("test", "value")
    print("value set")

async def test():
    client = RedisCluster.from_url(
        "redis://127.0.0.1:7001/0",
        max_connections=1,
        connection_error_retry_attempts=100,
    )
    await asyncio.gather(*[subtest(client) for _ in range(10)])
    await client.aclose()

if __name__ == "__main__":
    asyncio.run(test())

This example sets maximum numbers of simultaneous connections to 1 and generous limit of 100 reconnection attempts. It then fires ten simultaneous async tasks that connect to cluster.

Expected behavior

Reconnects should be attempted and all tasks should complete successfully (eventually).

Actual behavior

MaxConnectionsError is raised immediately, without attempting any reconnects.

Traceback (most recent call last):
  File "/home/jmusilek/test/venv/lib/python3.11/site-packages/redis/asyncio/cluster.py", line 1006, in acquire_connection
    return self._free.popleft()
           ^^^^^^^^^^^^^^^^^^^^
IndexError: pop from an empty deque

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jmusilek/test/test_redis.py", line 89, in <module>
    asyncio.run(test())
  File "/usr/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/home/jmusilek/test/test_redis.py", line 84, in test
    await asyncio.gather(*[subtest(client) for _ in range(10)])
  File "/home/jmusilek/test/test_redis.py", line 74, in subtest
    await redis.set("test", "value")
  File "/home/jmusilek/test/venv/lib/python3.11/site-packages/redis/asyncio/cluster.py", line 749, in execute_command
    raise e
  File "/home/jmusilek/test/venv/lib/python3.11/site-packages/redis/asyncio/cluster.py", line 720, in execute_command
    ret = await self._execute_command(target_nodes[0], *args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmusilek/test/venv/lib/python3.11/site-packages/redis/asyncio/cluster.py", line 774, in _execute_command
    return await target_node.execute_command(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmusilek/test/venv/lib/python3.11/site-packages/redis/asyncio/cluster.py", line 1040, in execute_command
    connection = self.acquire_connection()
                 ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jmusilek/test/venv/lib/python3.11/site-packages/redis/asyncio/cluster.py", line 1013, in acquire_connection
    raise MaxConnectionsError()
redis.exceptions.MaxConnectionsError
Exception ignored in: <function StreamWriter.__del__ at 0x7ff25131d620>
Traceback (most recent call last):
  File "/usr/lib/python3.11/asyncio/streams.py", line 395, in __del__
  File "/usr/lib/python3.11/asyncio/streams.py", line 343, in close
  File "/usr/lib/python3.11/asyncio/selector_events.py", line 860, in close
  File "/usr/lib/python3.11/asyncio/base_events.py", line 761, in call_soon
  File "/usr/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed
RuntimeError: Event loop is closed

Context

Documentation on Async Cluster Client states (emphasis is mine):

connection_error_retry_attempts (int, default: 3) – Number of times to retry before reinitializing when TimeoutError or ConnectionError are encountered. The default backoff strategy will be set if Retry object is not passed (see default_backoff in backoff.py). To change it, pass a custom Retry object using the “retry” keyword.

max_connections (int, default: 2147483648) – Maximum number of connections per node. If there are no free connections & the maximum number of connections are already created, a MaxConnectionsError is raised. This error may be retried as defined by connection_error_retry_attempts

This strongly suggests that reconnection should be attempted when max_connections is reached.