msgspec icon indicating copy to clipboard operation
msgspec copied to clipboard

Struct fields aren't discovered in Python 3.14

Open thatch opened this issue 6 months ago • 3 comments

Description

It looks like something is fundamentally broken in how Struct subclasses find their fields under 3.14b1. This was observed on Arm Mac with the latest release, and Linux with latest release as well as main.

Working, on 3.13:

[tim@tangerine ~]$ uvx --from='git+https://github.com/jcrist/msgspec@main' python
      Built msgspec @ git+https://github.com/jcrist/msgspec@bc60e96772c5e8a3babff967d86a9e7dfcdbfb1b
Installed 1 package in 1ms
Python 3.13.2 (main, Feb 12 2025, 14:51:17) [Clang 19.1.6 ] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from msgspec import Struct
>>> class S(Struct):
...     rule_name: str
...     
>>> S("foo")
S(rule_name='foo')
>>> S()
Traceback (most recent call last):
  File "<python-input-3>", line 1, in <module>
    S()
    ~^^
TypeError: Missing required argument 'rule_name'

Broken, on 3.14:

[tim@tangerine ~]$ uvx -p 3.14 --from='git+https://github.com/jcrist/msgspec@main' python
Installed 1 package in 3ms
Python 3.14.0b1 (main, May 14 2025, 15:26:05) [GCC 15.1.1 20250425] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from msgspec import Struct
>>> class S(Struct):
...     rule_name: str
...     
>>> S("foo")
Traceback (most recent call last):
  File "<python-input-2>", line 1, in <module>
    S("foo")
    ~^^^^^^^
TypeError: Extra positional arguments provided
>>> S()
S()

cc @nedbat

thatch avatar May 14 '25 15:05 thatch

Probably a dupe of #795 (but I skimmed over that one when searching, I didn't know this was annotations-related).

thatch avatar May 14 '25 15:05 thatch

I'm assuming this might be because of PEP 649.

If that's the case, your code should work if you add a from __future__ import annotations to the top of the module, as that future directive is supposed to be kept in tact:

In Python 3.14, the behavior of code using from __future__ import annotations is unchanged.

provinzkraut avatar May 14 '25 16:05 provinzkraut

Add from __future__ import annotations

And tests pass!

thatch avatar May 14 '25 17:05 thatch

Underlying issue is #651 Submitted PR is #852

ncoghlan avatar Aug 19 '25 12:08 ncoghlan

I investigated a few options to see if this could be worked around without relying on from __future__ import annotations (as that's sufficient for a self-contained application code base, but not ideal for a library API that may receive struct instances defined elsewhere).

  • overriding __init__ in a subclass: disallowed by msgspec.Struct
  • overriding __init_subclass__ in a subclass: no apparent effect (metaclass is presumably trying to extract the fields before it runs __init_subclass__)
  • using a subclass of type(msgspec.Struct) as the metaclass: disallowed by type(msgspec.Struct) (the flag that allows subclassing is cleared)

The underlying issue comes from reading cls.__dict__["__annotations__"] rather than cls.__annotations__, so Python 3.14's backwards compatibility mechanism for the migration to lazy annotation evaluation as the default doesn't get a chance to trigger (the PR in #852 adds the now required lazy annotation evaluation logic to msgspec's own internal annotation lookup rather than changing the annotation lookup to go through the class descriptor machinery, since the latter would be a more intrusive change).

Edit: I had also tried __init_subclass__, but hadn't noted here that it didn't offer a workaround either

ncoghlan avatar Aug 22 '25 02:08 ncoghlan

Resolved by https://github.com/jcrist/msgspec/pull/852

ofek avatar Oct 19 '25 23:10 ofek

https://github.com/jcrist/msgspec/releases/tag/0.20.0

ofek avatar Nov 24 '25 04:11 ofek