Compatibility with marshmallow 4.0.0

[astral-alpaca]

marshmallow-dataclass is not compatible with marshmallow 4.0.0. Here is a dataclass that fails with marshmallow 4.0.0:

@mmd.dataclass(frozen=True)
class Document:
    ...
    partnernummer: str | None = field(default=None, metadata={"is_partner_field": True})
    ...

This raises an exception when loading a calling Schema().load():

/code/venv/lib/python3.12/site-packages/marshmallow_dataclass/lazy_class_attribute.py:36: in __get__
    setattr(cls, self.name, self.func())
/code/venv/lib/python3.12/site-packages/marshmallow_dataclass/__init__.py:464: in class_schema
    return _internal_class_schema(clazz, base_schema)
/code/venv/lib/python3.12/site-packages/marshmallow_dataclass/__init__.py:571: in _internal_class_schema
    attributes.update(
/code/venv/lib/python3.12/site-packages/marshmallow_dataclass/__init__.py:574: in <genexpr>
    _field_for_schema(
/code/venv/lib/python3.12/site-packages/marshmallow_dataclass/__init__.py:919: in _field_for_schema
    union_field = _field_for_union_type(typ, base_schema, **metadata)
/code/venv/lib/python3.12/site-packages/marshmallow_dataclass/__init__.py:765: in _field_for_union_type
    return _field_for_schema(
/code/venv/lib/python3.12/site-packages/marshmallow_dataclass/__init__.py:862: in _field_for_schema
    return field(**metadata)
E   TypeError: Field.__init__() got an unexpected keyword argument 'is_partner_field'

Also, when dumping a dataclass i get another exception (Unfortunately no stack trace here):

    dumped = Document.Schema().dump(document)
E   TypeError: 'str' object is not callable

Marshmallow release notes for 4.0.0 are here: https://marshmallow.readthedocs.io/en/latest/changelog.html#id2

[dairiki]

What's going on, at least in your load() example is that, as of marshmallow 4.0, marshmallow.field.Field no longer puts unrecognized keyword contructor arguments into its .metadata dict. Instead it is expect that field metadata be passed in an explicit metadata argument to the constructor.

Admitedly at bit confusing, this means, that usage with marshmallow_dataclass requires a "nested" metadata key in the dataclasses.field metadata parameter.

I.e. this should work with any modern version of Marshmallow:

class Document:
    ...
    partnernummer: str | None = field(
        default=None,
        metadata={
            "metadata": {"is_partner_field": True}
        },
    )
    ...

I'm unable to reproduce your 'str' object is not callable error, so I'm not sure what that is. I suspect it's related to the above, however.

---

I can't think of a way for us (marshmallow_dataclass) to make this less confusing. We are not in controll of the naming of the parameters of dataclasses.field or of marshmallow.fields.Field. Both use a parameter named metadata for somewhat differing purposes. For us to try to figure out which bits of the dataclasses.field metadata is meant as a kw arg to Field, and which should be passed to its metadata parameter is too much magic, I think. (I suppose we could do it using inspect.signature, but it seems that that level of magic would probably add to the confusion.)

[dwt]

Sounds like using names like dataclass_metadata and marshmallow_metadata or field_metadata could be less confusing.

[dairiki]

Sounds like using names like dataclass_metadata and marshmallow_metadata or field_metadata could be less confusing.

It could be, however we have no say regarding the parameter names of dataclasses.field or marshmallow.fields.Field.

[dwt]

@dairiki Don't we hand these parameters over? I would have expected that renaming them on passing them on should be fairly simple?

[dairiki]

@dairiki Don't we hand these parameters over? I would have expected that renaming them on passing them on should be fairly simple?

Currently, we pass the whole mapping passed to dataclasses.field in its metadata parameter on, unmolested, as keyword parameters to the marshmallow field constructor.

The metadata parameter is the only parameter of dataclasses.field that accepts arbitrary data (unrelated to the working of the dataclass). Similarly, the metadata parameter of the marshmallow field constructor is (as of marshmallow 4) the only parameter that accepts arbitrary data (unrelated to the working of the field). We have no control over the names of either of those parameters.

We could, I suppose, mangle whatever mapping is passed to the dataclass field before passing it on to the marshmallow field constructor (e.g. rename any marshmallow_metadata key to metadata). To me, that seems unlikely to improve the net confusion budget.

We could also define our own marshmallow_dataclass.field to remap parameters to dataclasses.field. Again, that seems unlikely to reduce overall confusion.

[cmpilato]

If I understand correctly, the OP's problem would also manifest for more direct uses of @add_schema on regular ol' dataclasses, yes?

[marcdexet-cnrs]

Hi folks

I got the same problem with marshmallow 4.0.1

This breaks

    @dataclass
    class Level2(Level0):
        address: str | None = field(metadata={"index": True}, default=None)

This doesn't break

    @dataclass
    class Level2(Level0):
        address: str | None = field(metadata={'metadata': {"index": True}}, default=None)