aiohttp icon indicating copy to clipboard operation
aiohttp copied to clipboard

Fix invalid utf8 decoding error handler in HeadersParser

Open OttoWinter opened this issue 3 years ago • 1 comments

What do these changes do?

The xmlcharrefreplace error handler is invalid for decode operations (only encode):

In [1]: b = b"\xc3\x28"  # invalid utf8

In [2]: b.decode("utf8", "xmlcharrefreplace")
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-6-0fb1f7b80b46> in <module>
----> 1 b.decode("utf8", "xmlcharrefreplace")

TypeError: don't know how to handle UnicodeDecodeError in error callback

In [3]: b.decode("utf8", "surrogateescape")
Out[3]: '\udcc3('

When these LineTooLong exceptions are triggered with non-utf8 input, a TypeError exception was raised

This was found by atheris

Are there changes in behavior for the user?

No.

Related issue number

Checklist

  • [x] I think the code is well written
  • [ ] Unit tests for the changes exist
  • [ ] Documentation reflects the changes
  • [x] If you provide code modification, please add yourself to CONTRIBUTORS.txt
    • The format is <Name> <Surname>.
    • Please keep alphabetical order, the file is sorted by names.
  • [x] Add a new news fragment into the CHANGES folder
    • name it <issue_id>.<type> for example (588.bugfix)
    • if you don't have an issue_id change it to the pr id after creating the pr
    • ensure type is one of the following:
      • .feature: Signifying a new feature.
      • .bugfix: Signifying a bug fix.
      • .doc: Signifying a documentation improvement.
      • .removal: Signifying a deprecation or removal of public API.
      • .misc: A ticket has been closed, but it is not of interest to users.
    • Make sure to use full sentences with correct case and punctuation, for example: "Fix issue with non-ascii contents in doctest text files."

OttoWinter avatar Feb 13 '22 22:02 OttoWinter