taskiq icon indicating copy to clipboard operation
taskiq copied to clipboard

High CPU When Message Queue Unavailable

Open TheTechromancer opened this issue 5 months ago • 0 comments

When the message queue isn't reachable (i.e. Redis is offline), the receiver retries so quickly that the CPU goes to 100%. This should be an easy fix, such as adding a .1 second delay before retry.

Exception found while listening to the broker. unhandled errors in a TaskGroup (1 sub-exception)
  + Exception Group Traceback (most recent call last):
  |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/taskiq/api/receiver.py", line 80, in run_receiver_task
  |     await receiver.listen(finish_event)
  |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/taskiq/receiver/receiver.py", line 336, in listen
  |     async with anyio.create_task_group() as gr:
  |                ~~~~~~~~~~~~~~~~~~~~~~~^^
  |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/anyio/_backends/_asyncio.py", line 772, in __aexit__
  |     raise BaseExceptionGroup(
  |         "unhandled errors in a TaskGroup", self._exceptions
  |     ) from None
  | ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/redis/asyncio/connection.py", line 290, in connect
    |     await self.retry.call_with_retry(
    |         lambda: self._connect(), lambda error: self.disconnect()
    |     )
    |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/redis/asyncio/retry.py", line 59, in call_with_retry
    |     return await do()
    |            ^^^^^^^^^^
    |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/redis/asyncio/connection.py", line 723, in _connect
    |     reader, writer = await asyncio.open_connection(
    |                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |         **self._connection_arguments()
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |     )
    |     ^
    |   File "/usr/lib/python3.13/asyncio/streams.py", line 48, in open_connection
    |     transport, _ = await loop.create_connection(
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |         lambda: protocol, host, port, **kwds)
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/lib/python3.13/asyncio/base_events.py", line 1160, in create_connection
    |     raise exceptions[0]
    |   File "/usr/lib/python3.13/asyncio/base_events.py", line 1135, in create_connection
    |     sock = await self._connect_sock(
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^
    |         exceptions, addrinfo, laddr_infos)
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/usr/lib/python3.13/asyncio/base_events.py", line 1038, in _connect_sock
    |     await self.sock_connect(sock, address)
    |   File "/usr/lib/python3.13/asyncio/selector_events.py", line 641, in sock_connect
    |     return await fut
    |            ^^^^^^^^^
    |   File "/usr/lib/python3.13/asyncio/selector_events.py", line 681, in _sock_connect_cb
    |     raise OSError(err, f'Connect call failed {address}')
    | ConnectionRefusedError: [Errno 111] Connect call failed ('127.0.0.1', 6379)
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/taskiq/receiver/receiver.py", line 383, in prefetcher
    |     message = current_message.result()
    |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/taskiq_redis/redis_broker.py", line 278, in listen
    |     fetched = await redis_conn.xreadgroup(
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |     ...<9 lines>...
    |     )
    |     ^
    |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/redis/asyncio/client.py", line 641, in execute_command
    |     conn = self.connection or await pool.get_connection()
    |                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/redis/asyncio/connection.py", line 1280, in get_connection
    |     await self.ensure_connection(connection)
    |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/redis/asyncio/connection.py", line 1133, in ensure_connection
    |     await connection.connect()
    |   File "/home/user/.cache/pypoetry/virtualenvs/bbot-server-DmHQwTaK-py3.13/lib/python3.13/site-packages/redis/asyncio/connection.py", line 298, in connect
    |     raise ConnectionError(self._error_message(e))
    | redis.exceptions.ConnectionError: Error 111 connecting to localhost:6379. Connect call failed ('127.0.0.1', 6379).
    +------------------------------------

TheTechromancer avatar Jul 03 '25 14:07 TheTechromancer