cattrs

cattrs copied to clipboard

• cattrs version: 23.2.3
• Python version: 3.9
• Operating System: macOS

Description

I'm not sure if this idea has already come up somewhere, or if there's perhaps a much smarter/different way to approach the problem, but I wanted to ask if there's already a mechanism for specifying alternative constructors for structuring.

Consider the following class:

@define
class Point:
    x: float
    y: float

    @classmethod
    def from_polar(cls, radius: float, angle: float) -> Point:
        return Point(radius * math.cos(angle), radius * math.sin(angle))

Let's assume it is used somewhere down the line a nested object hierarchy, and potentially that the class could have many other classmethod constructors. I am looking for a way to specify the constructor in the structuring input.

What I Did

Here is a relatively simple hook that does the job

def select_constructor_hook(specs: dict, cls: Type[_T]) -> _T:
    """Use the constructor specified in the 'constructor' field for deserialization."""
    # If a constructor is specified, use it
    specs = specs.copy()
    if constructor_name := specs.pop("constructor", None):
        constructor = getattr(cls, constructor_name)

        # Extract the constructor parameter types and deserialize the arguments
        type_hints = get_type_hints(constructor)
        for key, value in specs.items():
            annotation = type_hints[key]
            specs[key] = cattrs.structure(specs[key], annotation)

        # Call the constructor with the deserialized arguments
        return constructor(**specs)

    # Otherwise, use the regular __init__ method
    return cattrs.structure_attrs_fromdict(specs, cls)

It allows to create an object as follows

cattrs.register_structure_hook(Point, select_constructor_hook)
cattrs.structure({"constructor": "from_polar", "radius": 1.0, "angle": math.pi}, Point)

The Question

To me, this appears like a rather common use case and I am wondering if there is already a similar/better mechanism in place. If not, would this be a pattern worth to be included into cattrs, potentially in the form of a strategy? Of course, the exact details would be need to fleshed out and things should become configurable (e.g. which keyword to use for the constructor specification, etc).

I could imagine something like

from cattrs.strategies import switch_constructors
switch_constructors("constructor", converter)

Happy to hear your thoughts 😃

Have you seen https://catt.rs/en/stable/strategies.html#using-class-specific-structure-and-unstructure-methods? Feels like it matches this use case.

Here's how this would look like using that strategy:

from __future__ import annotations

import math

from attrs import define

from cattrs import Converter
from cattrs.strategies import use_class_methods


@define
class Point:
    x: float
    y: float

    @classmethod
    def from_polar(cls, input: dict) -> Point:
        radius = float(input["radius"])
        angle = float(input["angle"])
        return Point(radius * math.cos(angle), radius * math.sin(angle))


c = Converter()

use_class_methods(c, "from_polar")

print(c.structure({"radius": 1.0, "angle": math.pi}, Point))

It's a little different since this version of from_polar takes a dict instead of the nice args.

If you only want this behavior on a single class, you could even simplify it further and avoid the strategy:

c.register_structure_hook(Point, Point.from_polar)

Your approach has the benefit of a much nicer class method signature. We can brainstorm how something like this would be adaptible to cattrs.

I got a little inspired and wanted to explore this idea I had. My idea was that a signature for an alternate constructor method could be expressed as a TypedDict, and cattrs can already structure typed dicts easily.

Here's a snippet:

from __future__ import annotations

import math
from inspect import signature
from typing import Callable, TypedDict

from attrs import define

from cattrs import Converter
from cattrs.dispatch import StructureHook


@define
class Point:
    x: float
    y: float

    @classmethod
    def from_polar(cls, radius: float, angle: float) -> Point:
        return Point(radius * math.cos(angle), radius * math.sin(angle))


c = Converter()


def signature_to_typed_dict(fn: Callable) -> type[TypedDict]:
    params = {p: t.annotation for p, t in signature(fn).parameters.items()}
    return TypedDict(f"{fn.__name__}_args", params)


def make_initializer_from(fn: Callable, conv: Converter) -> StructureHook:
    td = signature_to_typed_dict(fn)
    td_hook = conv.get_structure_hook(td)

    return lambda v, _: fn(**td_hook(v, td))


c.register_structure_hook(Point, make_initializer_from(Point.from_polar, c))
print(c.structure({"radius": "1.0", "angle": math.pi}, Point))

(I'm using converter.get_structure_hook which is from 24.1, unreleased as of yet).

Hi @Tinche. First of all, thanks for your quick and detailed answers, very much appreciated 👌🏼 And yes, I am aware of the use_class_methods approach – in fact, I've been using cattrs quite extensively over the last year or so and like the framework very much, great work BTW!

So here my detailed feedback:

• Using the built-in ability to structure TypedDicts is a great idea! This effectively simplifies my approach in that it removes the need of having an explicit for loop that handles structuring of the individual arguments 🥇
• Also, it is much better than your previous approach which required changing the method signature. The latter is not a realistic option in most use cases I'd say because, after all, the beauty of cattrs is that it decouples serialization from the class. That is, the class definition should always come first, and then cattrs needs to handle it as is. In fact, I could not easily modify the method signature in my use case since it would result in a breaking change.
• I wonder if there is a way how also positional-only arguments could be handled, or even a combination of positional-only and keyword arguments. Note: This is absolutely not high-priority (AFAIK they are not even available in attrs these days) and we can definitely postpone the discussion.

But here the most important part

I think you might have missed the point of my proposed approach, which is also the reason why I think none of the existing strategies can be used. The crux of the described use case lies in the fact that the selection of the constructor cannot be static but must be configurable at runtime. There are possibly more situations where this might be required but my specific context: I deserialize the object behind an API and need to enable to let the user specify the creation mechanism as part of the JSON they provide. What are your thoughts?

The crux of the described use case lies in the fact that the selection of the constructor cannot be static but must be configurable at runtime.

Interesting. I guess in a way this would be like a tagged union but for constructors. I think it's a cool idea, unsure how to package it into something in cattrs just yet.

So I guess there are several ways we can go about this:

1. If you think a strategy could be the right approach, I’d be happy to draft a PR with the TypedDict approach and we can iterate from there.
2. If you are unsure from what angle to approach it, we can put it in hold for a week or two and continue the discussion later.
3. And finally, if you come to the conclusion that this is our of scope for cattrs, I’ll just keep my custom solution. However, I really think the mechanism is general enough so that potentially many use could benefit from it.

Just let me know what you prefer ✌️

There's also a way where we invent something new, like a recipes section in the user manual. Think that might be a good way to go for things we're unsure should be strategies.

Closing since the docs PR was merged.