msgspec icon indicating copy to clipboard operation
msgspec copied to clipboard

Types inheriting from GenericAlias types (such as `Mapping`) cannot be decoded.

Open mwaddoups opened this issue 1 month ago • 3 comments

Description

If you try to make a generic class that inherits from one of the _SpecialGenericAlias types (such as Mapping), the type inference fails to work.

A minimal example below (Python 3.13, msgspec 0.20.0):

from collections.abc import Mapping
from dataclasses import dataclass

import msgspec


@dataclass
class Bar[T](Mapping[str, T]):
    data: dict[str, T]

    def __getitem__(self, x):
        return self.data[x]

    def __len__(self):
        return len(self.data)

    def __iter__(self):
        return iter(self.data)


x: Bar[int] = Bar({"x": 3})

msgspec.msgpack.decode(msgspec.msgpack.encode(x), type=Bar[int])

with traceback

Traceback (most recent call last):
  File "<censored>", line 31, in <module>
    msgspec.msgpack.decode(msgspec.msgpack.encode(x), type=Bar[int])
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/msgspec/_utils.py", line 252, in get_dataclass_info
    hints = get_class_annotations(obj)
  File "/opt/venv/lib/python3.13/site-packages/msgspec/_utils.py", line 149, in get_class_annotations
    mro, typevar_mappings = _get_class_mro_and_typevar_mappings(obj)
                            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/opt/venv/lib/python3.13/site-packages/msgspec/_utils.py", line 126, in _get_class_mro_and_typevar_mappings
    inner(obj, {})
    ~~~~~^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/msgspec/_utils.py", line 124, in inner
    inner(b, new_scope)
    ~~~~~^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.13/site-packages/msgspec/_utils.py", line 116, in inner
    params = cls.__parameters__
             ^^^^^^^^^^^^^^^^^^
AttributeError: type object 'Mapping' has no attribute '__parameters__'

I believe this is caused by the logic in inner(...) in the utils. For this, we look up __origin__ - however, in these specific cases, the types library has typing.Mapping as a typing._SpecialGenericAlias, which has different behaviour - it's __origin__ is not typing.Mapping, it's collections.abc.Mapping. This has no generic type parameters so inference fails.

This looks extremely tough to wade through so not sure what a good workaround is here - not a typing expert!

I'll note briefly that this behaviour goes away if it's not generic in inheritance, but it still fails if I use msgspec.Struct and a combined metaclass from StructMeta and ABCMeta so is not an artifact of dataclasses or anything, I think.

mwaddoups avatar Nov 26 '25 13:11 mwaddoups

This looks extremely tough to wade through so not sure what a good workaround is here - not a typing expert!

I've been looking at this and I agree. This is a tough one! But I think I've got it figured out, I'll submit a PR soon(ish).

There are lots of things at play here. At first I thought it was an easy fix, because we just need to fix the inspection. But fixing that uncovered a whole bunch of other bugs that needed fixing in order to make your example work.

  1. When subscribing a "new style" generic (such as collections.abc.Mapping), it produces a types.GenericAlias (vs. the "old style" typing._GenericAlias), which msgspec did not handle correctly during inspection
  2. When dealing with complex generics, msgspec did not account for type var syntax at all, so it couldn't get a proper resolution for these types
  3. During type conversion, msgspec caches certain information on the type objects themselves. For regular generics, this works because typing._GenericAlias has a __dict__, so you can just assign attributes to it. types.GenericAlias however has __slots__, so we can't cache our type information on there

provinzkraut avatar Nov 29 '25 12:11 provinzkraut

@mwaddoups I've put up a PR at #962. Can you confirm that (with msgspec installed from that branch) this fixes your original issue?

provinzkraut avatar Nov 29 '25 13:11 provinzkraut

Happy to confirm that branch solves both the toy issue described in our situation, as well as our original slightly more complex use case.

I glanced at the PR in the forlorn hope that I could understand the details, it remained a hope, but thanks for looking into this one!

mwaddoups avatar Nov 30 '25 15:11 mwaddoups