msgspec
msgspec copied to clipboard
jcrist
Union of str and decimal
Description
Hello, I have a situation where I am getting updates from the server in msgpack format, one of the fields is value that can be Union of str, int, float, bool and newly decimal. Looking through releases I see that you added support for encoding/decoding decimals but Union of str encoded types is not supported.
Do you have any tip how to surpass that? I thought custom extension would work, but since decimal is one of supported types now, I am not able to write extension for it.
Thank you
As a temporary solution only thing I could think of was to do something like this:
from typing import Any
from _decimal import Decimal as BuiltInDecimal
import msgspec
class Decimal(BuiltInDecimal):
"""Unfortunate hack to make msgspec serialize Decimal objects as extension type not as string."""
DECIMAL_TYPE_CODE = 17
def enc_hook(obj: Any) -> Any:
if isinstance(obj, Decimal):
encoded_dec = str(obj).encode("utf-8")
data = struct.pack(f"{len(encoded_dec)}s", encoded_dec)
# Return an `Ext` object so msgspec serializes it as an extension type.
return msgspec.msgpack.Ext(DECIMAL_TYPE_CODE, data)
# Raise a TypeError for other types
raise TypeError(f"Objects of type {type(obj)} are not supported")
def ext_hook(code: int, data: memoryview) -> Any:
if code == DECIMAL_TYPE_CODE:
dec_string = struct.unpack(f"{data.nbytes}s", data)[0].decode("utf-8")
return Decimal(dec_string)
# Raise a TypeError for other extension type codes
raise TypeError(f"Extension type code {code} is not supported")
class A(msgspec.Struct, array_like=True):
value: Any # str | Decimal | int | float | bool | None
DECODER = msgspec.msgpack.Decoder(type=A, ext_hook=ext_hook)
ENCODER = msgspec.msgpack.Encoder(enc_hook=enc_hook)
Just adding to this discussion, we like that msgspec handles decimal however they are embedded as strings. Here is some finding from another discussion around handling decimals.
https://github.com/meltano/sdk/issues/1046#issuecomment-1535493163
Do we think that msgspec might output decimals like simplejson without the double quotes in the future?
