Fix segmentation fault on Python 3.13
Fix for https://github.com/jcrist/msgspec/issues/910 and https://github.com/jcrist/msgspec/issues/868
The problem is only with Python 3.13, in Python 3.14 PyObject_GC_New does this initialization itself, see https://github.com/python/cpython/pull/123192
I'm curious though how you tracked it down to this?
It was difficult :) The short version:
From gdb bt found following calls:
4 make_dict_from_instance_attributes
#5 _PyObject_MaterializeManagedDict_LockHeld
#6 _PyObject_MaterializeManagedDict
#7 ensure_managed_dict
In next step, I found that values was not initialized:
(gdb) frame 4
#4 0x00000000023d7c59 in make_dict_from_instance_attributes (interp=0x2fcc2e8 <_PyRuntime+104400>, keys=0x7ffff4e24f20, values=0x7ffff54dc070) at /home/shadchin/arc/arcadia/contrib/tools/python3/Objects/dictobject.c:6736
6736 track += _PyObject_GC_MAY_BE_TRACKED(val);
(gdb) info locals
val = <unknown at remote 0xcdcdcdcdcdcdcdcd>
i = 0
used = 1
track = 0
size = 30
res = 0x7ffff1b535b0
(gdb) print values
$1 = (PyDictValues *) 0x7ffff54dc070
(gdb) print keys
$2 = (PyDictKeysObject *) 0x7ffff4e24f20
(gdb) x/10gx values
0x7ffff54dc070: 0xcdcdcdcdcdcdcdcd 0xcdcdcdcdcdcdcdcd
0x7ffff54dc080: 0xcdcdcdcdcdcdcdcd 0xcdcdcdcdcdcdcdcd
0x7ffff54dc090: 0xcdcdcdcdcdcdcdcd 0xcdcdcdcdcdcdcdcd
0x7ffff54dc0a0: 0xcdcdcdcdcdcdcdcd 0xcdcdcdcdcdcdcdcd
0x7ffff54dc0b0: 0xcdcdcdcdcdcdcdcd 0xcdcdcdcdcdcdcdcd
Then I had to study what managed dict/PyDictValues and Struct_alloc. I found that it's true for 3.13 that we don't initialize PyDictValues, looked at the CPython code, how it does it, found the _PyObject_InitInlineValues call.
Do we know why 3.14 is fine?
@shadchin seems like _PyObject_InitInlineValues isn't available?
https://github.com/jcrist/msgspec/actions/runs/19736064121/job/56566395047?pr=960#step:8:228
Do we know why 3.14 is fine?
Python 3.14+ does this initialization itself - https://github.com/python/cpython/blob/main/Python/gc.c#L2377-L2379
@shadchin seems like
_PyObject_InitInlineValuesisn't available?https://github.com/jcrist/msgspec/actions/runs/19736064121/job/56566395047?pr=960#step:8:228
We have our own build system, fix is working for us, but I forgot to check on a clean environment, sorry. I tried to make a fix but it didn't help. There's a conflict with write_u64.
I'll go upstream with it.
I took a different approach and it helped)
Maybe something else will prompt in upstream.