Possible to serialize a top-level list/array?

[ShardPhoenix]

JSON allows an array at top level (instead of an object). It would be nice if we have eg a List[MyDataClass] to be able to serialize this directly without a wrapper. Is this possible in mashumaro?

Currently I'm working around this like follows:

json = f"[{','.join([item.to_json() for item in my_list])}]"

[Fatal1ty]

Hi @ShardPhoenix

It's not possible with mashumaro at the moment. I'm thinking about a separated from dataclass function for serialization but it's not coming soon.

[illeatmyhat]

dataclasses-json has a many=true field for their dump() function. Seems to work well enough. https://github.com/lidatong/dataclasses-json#use-my-dataclass-with-json-arrays-or-objects

[Fatal1ty]

At the moment, it can be done using a generic dataclass wrapper. A couple of examples:

@dataclass
class CommonClass(
    Generic[T],
    DataClassJSONMixin,
    DataClassMessagePackMixin,
    DataClassYAMLMixin,
    DataClassTOMLMixin,
):
    x: T

    @classmethod
    def __pre_deserialize__(cls: Type[T], d: Dict[Any, Any]) -> Dict[Any, Any]:
        return {"x": d}

    def __post_serialize__(self: T, d: Dict[Any, Any]) -> Dict[Any, Any]:
        return d["x"]


def create_unpacker(data_type: T, fmt: Literal["dict", "json"] = "dict"):
    class ConcreteClass(CommonClass[data_type]):
        pass

    if fmt == "json":
        def unpacker(data) -> T:
            return ConcreteClass.from_json(data)
    else:
        def unpacker(data) -> T:
            return ConcreteClass.from_dict(data)

    return unpacker


def create_packer(data_type: T, fmt: Literal["dict", "json"] = "dict"):
    class ConcreteClass(CommonClass[data_type]):
        pass

    if fmt == "json":
        def packer(data) -> T:
            return ConcreteClass(data).to_json()
    else:
        def packer(data) -> T:
            return ConcreteClass(data).to_dict()

    return packer


@dataclass
class MyDataClass(DataClassDictMixin):
    x: datetime


my_json_packer = create_packer(list[MyDataClass], fmt="json")
print(repr(my_json_packer([MyDataClass(datetime.utcnow())])))
# '[{"x": "2023-03-26T10:33:44.567742"}]'

my_dict_packer = create_packer(list[MyDataClass], fmt="dict")
print(repr(my_dict_packer([MyDataClass(datetime.utcnow())])))
# [{'x': '2023-03-26T10:34:06.932352'}]


def from_json(cls: T, data) -> T:
    class ConcreteClass(CommonClass[cls]):
        pass

    return ConcreteClass.from_json(data).x


def as_json(cls: T, obj) -> T:
    class ConcreteClass(CommonClass[cls]):
        pass

    return ConcreteClass(obj).to_json()


print(repr(as_json(list[MyDataClass], [MyDataClass(datetime.utcnow())])))
# '[{"x": "2023-03-26T10:38:24.234213"}]'

print(from_json(list[MyDataClass], '[{"x": "2023-03-26T10:36:25.902384"}]'))
# [MyDataClass(x=datetime.datetime(2023, 3, 26, 10, 36, 25, 902384))]

[RA80533]

Is that specific construct something you were going to add to the package?

Is there anything I can do to help bring this feature to fruition?

[Fatal1ty]

Is that specific construct something you were going to add to the package?

I've given a couple of examples of how this issue can be addressed from outside, but I wasn't going to include it to the package because it's still a workaround. For native support I see the following steps:

• Come up with the convenient API
• Extend the existing CodeBuilder or create something similar to it that would build functions for specific types from ground up without extra dataclasses

Is there anything I can do to help bring this feature to fruition?

You can play around with CodeBuilder and pack.py / unpack.py modules and try to get a proof of concept. Any help is welcome.

[joaonc]

Started working with mashumaro and congrats on a great package. IMO, the ability to serialize as array at top level is a killer feature as it's used in many scenarios.

One suggestion is to add the functionality above into the package, even if it's not a complete solution and then modify it later, with the risk of breaking the API if a better solution comes along. This is much needed 😃

[Fatal1ty]

Good news everyone! I’m working on it, so this functionality will be a part of 3.11 release.

[Fatal1ty]

The long-awaited pull request has landed! See updated docs here.