bancho.py
bancho.py copied to clipboard
misc: determine a method to globally handle custom json serialization using `httpx`
At the moment, our server has a couple of locations where objects of types such as datetime are not able to be serialized into JSON due to a lack of support in the stdlib json module. httpx -- the http client we use for outgoing requests, relies specifically on the stdlib json module.
I see a couple of ways we can tackle this:
- Use
post(headers={'Content-Type': 'application/json'}, data=custom_json_dumps(...))in each call. Never usepost(json=...). Seems quite error prone/duplicative. - Subclass
AsyncClient, override thebuild_requestmethod, copy most fromhttpxbut use a customRequestwhich overrides__init__to perform option 1's suggestion automatically before yielding tosuper().__init__. - Monkey-patch
httpx._content.encode_jsonto provide a customJSONEncodersubclass tojson.dumps(cls=...). - Implement the
httpxmaintainer's proposal tohttpxto implement support forAsyncClient(request_class=..., response_class=...). Do the same__init__override as suggested in option 1.
My preference is probably option 4 -- seems the best for the overall ecosystem. We could potentially do one of the other options in the short term as I believe this is currently having an effect on production.
cc: @tsunyoku @NiceAesth @minisbett
potential hotfix:
import orjson
import httpx._content
from typing import Any, Tuple, Dict
def encode_json(json: Any) -> Tuple[Dict[str, str], httpx._content.ByteStream]:
body = orjson.dumps(json)
content_length = str(len(body))
content_type = "application/json"
headers = {"Content-Length": content_length, "Content-Type": content_type}
return headers, httpx._content.ByteStream(body)
httpx._content.encode_json = encode_json