httmock icon indicating copy to clipboard operation
httmock copied to clipboard

Support content encodings other than utf-8

Open michaelwooley opened this issue 1 year ago • 0 comments

The content attribute is always encoded as "utf-8" (source).

However, we've noticed that https://google.com/ (as of 9/29) now returns content encoded as 'ISO-8859-1'.

This breaks some tests that do an actual http request then play back the result and compare the content attributes.

If you're curious, here is a demo of what happens in a (non-mocked) google.com result:

import requests

url = "https://google.com/"
res = requests.get(url)

print(res.headers['Content-Type']) # 'text/html; charset=ISO-8859-1'
print(res.encoding)  # 'ISO-8859-1'

try:
    res.content.decode("utf8")
except Exception as e:
    print(e)

content = res.content.decode(res.encoding)

michaelwooley avatar Sep 29 '23 20:09 michaelwooley