faust icon indicating copy to clipboard operation
faust copied to clipboard

Polymorphic fields

Open gpkc opened this issue 2 years ago • 0 comments

I am in the process of converting a legacy system that was using an in-house streaming package to Faust. We had our own "polymorphic typing" implementation that made messages look like this:

{
    "class": "MessageClassType",
    "data": {"id": "1234", "other_field": "4321", ...}
}

I've been trying to match this to Faust's Polymorphic fields, but I'm stuck. From the documentation, I cannot understand if this feature should be used only when Faust has produced messages and added the blessed key, or if I can customize this. I am currently not yet producing messages using Faust, so it's unclear if Faust will always add the blessed key to the messages.

So far, I was able to come up with something like this:

class MyMessage(Record, polymorphic_fields=True, serializer='avroschema'):
    _blessed_key = 'blessed_key'

    class_: str

    @classmethod
    def from_data(cls, data: Mapping, **kwargs):
        if 'class' in data:
            data = {**data, "class_": data['class'], 'blessed_key': {"ns": data['class']}}
        return super().from_data(data, **kwargs)

class PolymorphicMessageField(Record):
    id: str
    other_field: str


class MyMessagePolymorphic(MyMessage, namespace='MyMessagePolymorphic'):
    class_ = "MyMessagePolymorphic"
    data: PolymorphicMessageField

This seems to be working. If I receive a message with the contents {"class": "MyMessagePolymorphic", "data": {"id": "1234", "other_field": "4321"}}, I am able to check the type on both the message (matching MyMessagePolymorphic), and the data field (matching PolymorphicMessageField).

However, this seems overly clunky for a couple reasons:

  • I am specifying the blessed key by overriding the underscore attribute _blessed_key.
  • I need to set the namespace to the same class name which causes repetition, otherwise Faust automatically appends the module name into it.
  • It is necessarily the case that the blessed key will be a dictionary with the class name inside a ns field, so I also need to override the from_data method.

Is there a better or more supported way to accomplish this?

gpkc avatar Mar 02 '23 18:03 gpkc