redis
Async retry needs to capture OSError exception in retry
Version: What redis-py and what redis version is the issue happening on?
redis-py 5.20
Platform: What platform / version? (For example Python 3.5.1 on Windows 7 / Ubuntu 15.10 / Azure)
Fedora 40, Python 3.12
Description: Description of your issue, stack traces from errors and code that reproduces the issue
I tried the program for asyncio version from Redis doc by taking down Redis server.
The first exception was raised is built-in OSError rather than redis.exceptions.ConnectionError. The exception regarding to the lost connection from Redis sever behave differently between the asnycio version and the normal version.
import asyncio
import redis.asyncio as redis
from redis.asyncio.retry import Retry
from redis.backoff import ExponentialBackoff
from redis.exceptions import BusyLoadingError, ConnectionError, TimeoutError
import logging
# Configure the logging module
logging.basicConfig(
format='%(asctime)s - %(levelname)s - %(message)s',
level=logging.INFO
)
async def main():
logging.info(f"Creating async Redis client...")
r = await redis.from_url("redis://127.0.0.1",
retry=Retry(ExponentialBackoff(8, 1), 25),
retry_on_error=[BusyLoadingError, ConnectionError, TimeoutError, ConnectionResetError, ])
logging.info(f"Created async Redis client...")
logging.info(f"Redis client pinging...")
await r.ping()
logging.info(f"Redis client pinged!")
logging.info(f"Closing async Redis client...")
await r.aclose()
logging.info(f"Closed async Redis client...")
# start the asyncio program
asyncio.run(main())
/home/Ricky/.virtualenv/pytool/bin/python /home/Ricky/private/repo/pytool/asyncio/demo_asyncio_redis_client_retry.py
2024-12-05 14:18:09,802 - INFO - Creating async Redis client...
2024-12-05 14:18:09,803 - INFO - Created async Redis client...
2024-12-05 14:18:09,803 - INFO - Redis client pinging...
Traceback (most recent call last):
File "/home/Ricky/.virtualenv/pytool/lib/python3.12/site-packages/redis/asyncio/connection.py", line 275, in connect
await self.retry.call_with_retry(
File "/home/Ricky/.virtualenv/pytool/lib/python3.12/site-packages/redis/asyncio/retry.py", line 59, in call_with_retry
return await do()
^^^^^^^^^^
File "/home/Ricky/.virtualenv/pytool/lib/python3.12/site-packages/redis/asyncio/connection.py", line 691, in _connect
reader, writer = await asyncio.open_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/asyncio/streams.py", line 48, in open_connection
transport, _ = await loop.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/asyncio/base_events.py", line 1121, in create_connection
raise exceptions[0]
File "/usr/lib64/python3.12/asyncio/base_events.py", line 1103, in create_connection
sock = await self._connect_sock(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/asyncio/base_events.py", line 1006, in _connect_sock
await self.sock_connect(sock, address)
File "/usr/lib64/python3.12/asyncio/selector_events.py", line 651, in sock_connect
return await fut
^^^^^^^^^
File "/usr/lib64/python3.12/asyncio/selector_events.py", line 691, in _sock_connect_cb
raise OSError(err, f'Connect call failed {address}')
ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 6379)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/Ricky/private/repo/pytool/asyncio/demo_asyncio_redis_client_retry.py", line 31, in <module>
asyncio.run(main())
File "/usr/lib64/python3.12/asyncio/runners.py", line 194, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/lib64/python3.12/asyncio/base_events.py", line 687, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File "/home/Ricky/private/repo/pytool/asyncio/demo_asyncio_redis_client_retry.py", line 23, in main
await r.ping()
File "/home/Ricky/.virtualenv/pytool/lib/python3.12/site-packages/redis/asyncio/client.py", line 611, in execute_command
conn = self.connection or await pool.get_connection(command_name, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/Ricky/.virtualenv/pytool/lib/python3.12/site-packages/redis/asyncio/connection.py", line 1058, in get_connection
await self.ensure_connection(connection)
File "/home/Ricky/.virtualenv/pytool/lib/python3.12/site-packages/redis/asyncio/connection.py", line 1091, in ensure_connection
await connection.connect()
File "/home/Ricky/.virtualenv/pytool/lib/python3.12/site-packages/redis/asyncio/connection.py", line 283, in connect
raise ConnectionError(self._error_message(e))
redis.exceptions.ConnectionError: Error 111 connecting to 127.0.0.1:6379. Connect call failed ('127.0.0.1', 6379).
Process finished with exit code 1
you can create a custom Retry class with adjusted call_with_retry() method and pass it down to pool/client:
class NeatRetry(Retry):
def __init__(self, backoff, retries: int, supported_errors):
if OSError not in supported_errors:
supported_errors += (OSError,)
super().__init__(backoff, retries, supported_errors)
async def call_with_retry(self, do: Callable[[], Awaitable[T]], fail: Callable[[RedisError], Any]) -> T:
"""
Execute an operation that might fail and returns its result, or
raise the exception that was thrown depending on the `Backoff` object.
`do`: the operation to call. Expects no argument.
`fail`: the failure handler, expects the last error that was thrown
"""
self._backoff.reset()
failures = 0
while True:
try:
return await do()
except self._supported_errors as error:
if isinstance(error, OSError) and error.args and len(error.args) > 0:
if error.args[0] not in (61, 104, 111):
# retrying ENODATA, ECONNRESET, ECONNREFUSED
raise error
failures += 1
await fail(error)
if self._retries >= 0 and failures > self._retries:
raise error
backoff = self._backoff.compute(failures)
if backoff > 0:
await asyncio.sleep(backoff)
Hi! Client instance accept a list of exceptions you want to Retry on, you can specify OSError there
retry_on_error: Optional[list] = None,
Closing issue for now. Let me know if it doesn't resolved
Why was this even closed @vladvildanov ? The provided response does not address the issue at hand. This behaviour needs to be documented somewhere.
I second your point, @1henrypage. It is a hidden bomb in the absence of proper documentation. The Asyncio version redis-py is not a drop-in replacement for the regular version.
@1henrypage You're right, was a bit confused with first response, looked like an issue with retries. Reopening issue