redis-py icon indicating copy to clipboard operation
redis-py copied to clipboard
verified publisher icon redis

Possible regression in 5.0.1: AttributeError: 'NoneType' object has no attribute 'feed'

Open PoByBolek opened this issue 2 years ago • 5 comments

Version

  • redis-py: 5.0.1
  • redis: 7.0.12, 7.2.1

Platform

  • Python 3.10.11 on Windows 11
  • Python 3.10.12 on Ubuntu 22.04 (WSL)

Description

This may be a duplicate of #2349, in which case I'd like to re-open that issue, but I got a similar error today with redis-py 5.0.1 (worked fine in 5.0.0).

Reading through the stack trace in #2349 again, this may be a separate issue after all? #2349 seemed to happen for async operations. Here it is a synchronous operation...

import logging
from threading import Thread
import time

from redis import Redis


_error = None

LOG = logging.getLogger(__name__)


def main() -> None:
    global _error

    logging.basicConfig(format='%(asctime)s - %(levelname)-8s - %(threadName)-10s - %(message)s')

    redis = Redis.from_url('redis://localhost')
    threads = [Thread(target=_worker, args=[redis, f'key:{i}']) for i in range(3)]
    for thread in threads:
        thread.start()

    try:
        while not _error:
            time.sleep(1)
    except KeyboardInterrupt as e:
        _error = e
    finally:
        for thread in threads:
            thread.join()


def _worker(redis: Redis, key: str) -> None:
    global _error

    round_number = 0

    try:
        while not _error:
            round_number += 1
            with redis:
                redis.setnx(key, 12)
    except Exception as e:
        _error = e
        LOG.exception('Failed in round %d', round_number)


if __name__ == '__main__':
    main()

Again, this works fine using redis-py 5.0.0 (even with 20 or more threads). But with 5.0.1 on Windows I get either an AttributeError:

2023-10-08 13:05:35,683 - ERROR    - Thread-2 (_worker) - Failed in round 1
Traceback (most recent call last):
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\connection.py", line 500, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\_parsers\hiredis.py", line 124, in read_response
    self.read_from_socket()
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\_parsers\hiredis.py", line 84, in read_from_socket
    bufflen = self._sock.recv_into(self._buffer)
OSError: [WinError 10038] An operation was attempted on something that is not a socket

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\PoByBolek\redis-bug\redis_bug.py", line 42, in _worker
    redis.setnx(key, 12)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\commands\core.py", line 2374, in setnx
    return self.execute_command("SETNX", name, value)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 536, in execute_command
    return conn.retry.call_with_retry(
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\retry.py", line 49, in call_with_retry
    fail(error)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 540, in <lambda>
    lambda error: self._disconnect_raise(conn, error),
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 526, in _disconnect_raise
    raise error
  File "C:\Users\PoByBolek\redis-bug\redis_bug.py", line 42, in _worker
    redis.setnx(key, 12)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\commands\core.py", line 2374, in setnx
    return self.execute_command("SETNX", name, value)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 536, in execute_command
    return conn.retry.call_with_retry(
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\retry.py", line 46, in call_with_retry
    return do()
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 537, in <lambda>
    lambda: self._send_command_parse_response(
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 513, in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 553, in parse_response
    response = connection.read_response()
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\connection.py", line 500, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\_parsers\hiredis.py", line 124, in read_response
    self.read_from_socket()
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\_parsers\hiredis.py", line 87, in read_from_socket
    self._reader.feed(self._buffer, 0, bufflen)
AttributeError: 'NoneType' object has no attribute 'feed'

or a ConnectionError:

2023-10-08 13:13:52,487 - ERROR    - Thread-1 (_worker) - Failed in round 2
Traceback (most recent call last):
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\connection.py", line 500, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\_parsers\hiredis.py", line 124, in read_response
    self.read_from_socket()
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\_parsers\hiredis.py", line 84, in read_from_socket
    bufflen = self._sock.recv_into(self._buffer)
OSError: [WinError 10038] An operation was attempted on something that is not a socket

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\PoByBolek\redis-bug\redis_bug.py", line 42, in _worker
    redis.setnx(key, 12)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\commands\core.py", line 2374, in setnx
    return self.execute_command("SETNX", name, value)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 536, in execute_command
    return conn.retry.call_with_retry(
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\retry.py", line 49, in call_with_retry
    fail(error)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 540, in <lambda>
    lambda error: self._disconnect_raise(conn, error),
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 526, in _disconnect_raise
    raise error
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\retry.py", line 46, in call_with_retry
    return do()
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 537, in <lambda>
    lambda: self._send_command_parse_response(
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 513, in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\client.py", line 553, in parse_response
    response = connection.read_response()
  File "C:\Users\PoByBolek\redis-bug\venv\lib\site-packages\redis\connection.py", line 508, in read_response
    raise ConnectionError(
redis.exceptions.ConnectionError: Error while reading from localhost:6379 : (10038, 'An operation was attempted on something that is not a socket', None, 10038, None)

On Ubuntu 22.04 (WSL) the same code produces a different ConnectionError (again this works fine with redis-py 5.0.0):

2023-10-08 13:19:50,274 - ERROR    - Thread-1 (_worker) - Failed in round 1
Traceback (most recent call last):
  File "/home/PoByBolek/redis-bug/redis_bug.py", line 42, in _worker
    redis.setnx(key, 12)
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/commands/core.py", line 2374, in setnx
    return self.execute_command("SETNX", name, value)
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/client.py", line 536, in execute_command
    return conn.retry.call_with_retry(
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/retry.py", line 49, in call_with_retry
    fail(error)
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/client.py", line 540, in <lambda>
    lambda error: self._disconnect_raise(conn, error),
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/client.py", line 526, in _disconnect_raise
    raise error
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/retry.py", line 46, in call_with_retry
    return do()
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/client.py", line 537, in <lambda>
    lambda: self._send_command_parse_response(
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/client.py", line 513, in _send_command_parse_response
    return self.parse_response(conn, command_name, **options)
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/client.py", line 553, in parse_response
    response = connection.read_response()
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/connection.py", line 500, in read_response
    response = self._parser.read_response(disable_decoding=disable_decoding)
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/_parsers/hiredis.py", line 124, in read_response
    self.read_from_socket()
  File "/home/PoByBolek/redis-bug/venv/lib/python3.10/site-packages/redis/_parsers/hiredis.py", line 86, in read_from_socket
    raise ConnectionError(SERVER_CLOSED_CONNECTION_ERROR)
redis.exceptions.ConnectionError: Connection closed by server.

PoByBolek avatar Sep 28 '23 09:09 PoByBolek

I've added some code that hopefully helps reproducing this issue in case anyone cares.

PoByBolek avatar Oct 08 '23 11:10 PoByBolek

This issue is marked stale. It will be closed in 30 days if it is not updated.

github-actions[bot] avatar Jun 21 '25 00:06 github-actions[bot]

Issue is reproducible with latest code - I was able to reproduce the last exception exampple.

petyaslavova avatar Jun 23 '25 09:06 petyaslavova

Reproduced with

hiredis                        3.2.1
redis                          5.3.1

alexon1234 avatar Sep 30 '25 07:09 alexon1234

Can confirm this happens when calling redis.close() or with redis: when using multi-threading with the same client. A possible workaround is to centralize the closing of the client after the thread.join(), but it would be nicer to have a proper exception, since current solution seems not-quite thread safe (for close()).

vogre avatar Nov 20 '25 08:11 vogre