aiohttp icon indicating copy to clipboard operation
aiohttp copied to clipboard

Error in StreamReader#readuntil

Open vitaliihonta opened this issue 3 years ago • 1 comments

Describe the bug

Hi there, it looks like StreamReader#readuntil function here has an error.

My original use case was to read CSV files directly from ClientResponse line-by-line when the line terminator is \r\n. The example I've provided is very similar.

When the separator has more than one symbol, it splits lines in a strange way. For instance, when separator is b'ab', it splits as follows:

b'11a'
b'b\n22a'
b'b\n33a'
b'b'

e.g. at the middle of the separator.

I think that this part wasn't updated at the time when it was readline function with only \n (one symbol length) separator available.

Probably this line should be:

ichar = self._buffer[0].find(separator, offset) + len(separator) 

so that it will contain the whole separator instead of 'splitting' it.

To Reproduce

Run this thing https://gist.github.com/vitaliihonta/b9be77deb0dfea1e9a411e695e6f15c9

Expected behavior

Each line should contain the whole separator

b'11ab'
b'\n22ab'
b'\n33ab'

Logs/tracebacks

None

Python Version

Python 3.8.9

aiohttp Version

3.8.1

multidict Version

Name: multidict
Version: 6.0.2

yarl Version

Name: yarl
Version: 1.7.2

OS

macOS

Related component

Client

Additional context

No response

Code of Conduct

  • [X] I agree to follow the aio-libs Code of Conduct

vitaliihonta avatar Apr 14 '22 09:04 vitaliihonta

Looks reasonable to me. If you can write a test and create a PR, that would be great.

Dreamsorcerer avatar Apr 15 '22 13:04 Dreamsorcerer