StreamReader.read(-1) cannot receive data before the write end is closed

[wangst321]

Echo client

import asyncio

loop = asyncio.get_event_loop()
coro = asyncio.open_connection('127.0.0.1', 4001)
reader, writer = loop.run_until_complete(coro)
writer.write(b'hello')
writer.write_eof()

# bug: it just return and cannot get echo_msg
msg = loop.run_until_complete(reader.read(-1))
print(msg)

Echo server

import asyncio

@asyncio.coroutine
def process_request(reader, writer):
    peername = writer.get_extra_info("peername")
    msg = yield from reader.read(-1)
    print(peername, msg)
    writer.write(msg)
    writer.write_eof()

loop = asyncio.get_event_loop()
coro = asyncio.start_server(process_request, host='127.0.0.1', port=4001)
server = loop.run_until_complete(coro)

try:
    loop.run_forever()
finally:
    loop.stop()

Run the server, then run the client. The client should get

b'hello'

but, it got

b''

the client didn't wait for the write end close.

msg = loop.run_until_complete(reader.read(-1))   # not work

My python version is

python3.4.3

[gvanrossum]

Interesting. I spent some time tracing through this code and haven't gotten further than observing that if you change the read(-1) in the server to read(1000) things work as expected. Now read(-1) calls read(self._limit) recursively, and this means that it effectively calls read(65536) twice, the second time returning EOF. I suspect that this causes some state to be lost. But I haven't found the bug yet.

[gvanrossum]

OK, I think I found it. StreamReaderProtocol.eof_received() should return True to keep the transport open even after receiving an EOF. I've filed a bug in the Python tracker: http://bugs.python.org/issue24539

[gvanrossum]

The fix is in https://github.com/python/asyncio/commit/ce3ad816a2ef9456b4b1c26b99dfc85ea1236811 but I'm keeping this issue (and the CPython issue) open because it needs a test and merging into CPython.

[gvanrossum]

It's now still open because (1) there's still no test AFAIK, and (2) the fix elicits a warning for SSL streams because those don't support write_eof().