aioredis-py icon indicating copy to clipboard operation
aioredis-py copied to clipboard

aioredis==1.3.1 & Redis 6.0.12 connection error with reader at end of file.

Open linux2000in opened this issue 3 years ago • 7 comments

After upgrading Redis to 6.0.12, now every time we trying to make a connection it closed the connection. it works perfectly with Redis = 5.4.10 and Aio 1.3.1

We are using tornado as app server and make call to redis using aioredis=1.3.1 , I always get 503 from tornado and get this error in tornado logs.

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/tornado/web.py", line 1592, in _execute
    result = yield result
  File "/usr/local/lib/python3.8/site-packages/tornado/gen.py", line 1133, in run
    value = future.result()
  File "/app/routes/entitlementsync.py", line 27, in post
    status = await self.redis.hget(entitlement_status_hash, "status")
aioredis.errors.ConnectionClosedError: Reader at end of file

let me know if you guys need more details.

linux2000in avatar Jul 06 '21 17:07 linux2000in

Are there any errors logged in Redis? What if you run MONITOR from the redis-cli while this happens?

abrookins avatar Jul 06 '21 17:07 abrookins

@abrookins - When I run the test ( which generates the error ) I was pinging in loop Redis using redis-cli and showlog does not give any error that the reason I came to the conclusion something wrong with aioredis module.

i also verified that when I see an error Redis connection is 100% available.

linux2000in avatar Jul 06 '21 18:07 linux2000in

Did more debug and found this. looks like aioredis issue based on docs it client loosing the conenction but WHY ? any suggestion how to debug more in details ?

severity: "ERROR" textPayload: "aioredis.errors.ConnectionClosedError: Reader at end of file"

File "/usr/local/lib/python3.8/site-packages/aioredis/util.py", line 66, in wait_make_dict"

Any idea what is going on ?

linux2000in avatar Jul 15 '21 18:07 linux2000in

@linux2000in are you using aioredis 1.3.1 or aioredis 2.0.0b1? We do not support aioredis 1.3.1 at all anymore.

Andrew-Chen-Wang avatar Jul 15 '21 18:07 Andrew-Chen-Wang

@Andrew-Chen-Wang - I am using aioredis . What is the latest stable version I should be using with enterprise Redis 6.x.x ?

linux2000in avatar Jul 15 '21 21:07 linux2000in

Unfortunately, the latest "stable" version is actually in beta, version 2.0.0b1, which we released yesterday. For aioredis==1.3.1, I have no idea why it's causing issues as the changelog for redis doesn't mention much: https://raw.githubusercontent.com/antirez/redis/6.0/00-RELEASENOTES

There was only a PR to fix the docs: Fix RedisModule_HashGet examples (#6697)

Do you mind installing aioredis==2.0.0b1 to see if that works out?

Andrew-Chen-Wang avatar Jul 15 '21 22:07 Andrew-Chen-Wang

More debug with aio==1.3.1

2021-07-16 22:33:20,824 aioredis DEBUG My AIOREDIS_DEBUG
2021-07-16 22:33:21,279 aioredis DEBUG Creating tcp connection to ('redis-11000.
redis-stage.europe-west4.gcp.xxxxx.net', '11000')
2021-07-16 22:46:57,255 aioredis DEBUG Connection has been closed by server, res
ponse: None
2021-07-16 22:46:57,255 aioredis DEBUG Cancelling waiter (<Future pending cb=[<T
askWakeupMethWrapper object at 0x7fbc4ec54c70>()]>, ['utf-8', None])
2021-07-16 22:46:58,930 aioredis DEBUG Creating tcp connection to ('redis-11000.
redis-stage.europe-west4.gcp.xxxx.net', '11000')
2021-07-16 22:47:19,180 aioredis DEBUG Connection has been closed by server, res
ponse: None
2021-07-16 22:47:19,180 aioredis DEBUG Cancelling waiter (<Future pending cb=[<T
askWakeupMethWrapper object at 0x7fbc4f4d9f70>()]>, ['utf-8', None])

linux2000in avatar Jul 16 '21 22:07 linux2000in