msgspec
msgspec copied to clipboard
Methods to convert to/from builtin objects
msgspec currently contains methods for converting converting objects to/from bytes using either JSON or MessagePack protocols. Sometimes it'd be useful to convert to/from "simpler" types (lists, dicts, ...).
Example use cases:
- Encoding using a third-party protocol library like
pyyaml. This can currently be handled on the encoding end by passing in a customdefaultmethod, but if the encode call is buried in some wrapper library the user may not have access to it directly and would need to recursively simplify the object before handing it off. - Decoding from a third-party protocol like
pyyamlinto higher-level types, while keeping the type validation. The easiest way to do this right now is to roundtrip the message through a supported protocol, for example:
def from_builtins(obj: Any, type: Type[T]) -> T:
msg = msgspec.json.encode(obj)
return msgspec.json.decode(msg, type=type)
Proposed initial API:
def to_builtins(obj: Any, *, recurse=True) -> Any:
"""Convert obj to simple builtin types (list, dict, ...). If `recurse` is True, this is applied recursively.
Note that copying only happens when necessary, if a list of integers is passed in the same list
will be returned."""
...
def from_builtins(obj: Any, type: Type[T]) -> T:
"""Convert obj to type. Note that copying only happens when necessary for conversion."""
...
I'm not 100% happy with this names, but all other options I could think of I liked less:
to_simple/from_simplesimplify/convert(I don't like the asymmetry in name here, andconvertimplies some casting behaviors likefloat -> intthat we won't actually support)cattrscalls thesedestructure/structurerespectivelylower/lift(for converting to lower/higher level types)
A few open questions:
- What types are valid to return from
to_builtins? My initial reaction isdict,list,tuple,str,int,float,bool,None(and no subclasses). - How do we handle types that encode/decode differently between JSON and msgpack? These would be
bytes/bytearrayanddatetimecurrently (we wouldn't supportExt/Rawin this API anyway)? Perhapsconvert_binary=Trueandconvert_datetime=Trueflags into_builtins?
If anyone has thoughts on how to spell these APIs, please let me know below.