PyO3
Metaissue: 3.15 support
This is to track test failures caused when compiling against 3.15. I know it's a bit early (even contradicting my own advice in CONTRIBUTING.md), but the sooner they're fixed before beta 1, the better.
- [ ] #5611
- [ ] #5612
- [ ] #5613
- [ ] #5614
@ngoldbaum mind taking a peek?
Basically all the erorrs are
SystemError: invalid PyModuleDef, extension possibly compiled for non-free-threaded Python
Maybe try nox -s ffi-check with a Python 3.15 alpha interpreter?
Alternatively this might be related to #4882 and the cache is broken.
https://github.com/python/cpython/commit/32b2c5de16183c74e43da01c5b33343e43c469a4 I think there might be an ABI change in PyModuleDef_HEAD_INIT?
With nox -s ffi-check:
(venv) @clin1234 ➜ /workspaces/pyo3 (3.15) $ nox -s ffi-check
nox > Running session ffi-check
nox > Creating virtual environment (virtualenv) using python3.15-nogil in .nox/ffi-check
nox > cargo doc --manifest-path=pyo3-ffi-check/Cargo.toml -p pyo3-ffi --no-deps
Compiling pyo3-build-config v0.27.1 (/workspaces/pyo3/pyo3-build-config)
Compiling pyo3-ffi v0.27.1 (/workspaces/pyo3/pyo3-ffi)
Documenting pyo3-ffi v0.27.1 (/workspaces/pyo3/pyo3-ffi)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 8.25s
Generated /workspaces/pyo3/pyo3-ffi-check/target/doc/pyo3_ffi/index.html
nox > cargo run --manifest-path=pyo3-ffi-check/Cargo.toml
Compiling pyo3-build-config v0.27.1 (/workspaces/pyo3/pyo3-build-config)
Compiling pyo3-ffi v0.27.1 (/workspaces/pyo3/pyo3-ffi)
Compiling pyo3-ffi-check v0.1.0 (/workspaces/pyo3/pyo3-ffi-check)
Compiling pyo3-ffi-check-macro v0.1.0 (/workspaces/pyo3/pyo3-ffi-check/macro)
Finished `dev` profile [unoptimized + debuginfo] target(s) in 29.85s
Running `pyo3-ffi-check/target/debug/pyo3-ffi-check`
comparing pyo3-ffi against headers generated for 3.15.0a1
warning: ignoring zero-sized pyo3_ffi type PyCodeObject
warning: ignoring zero-sized pyo3_ffi type PyDictKeysObject
warning: ignoring zero-sized pyo3_ffi type PyDictValues
warning: ignoring zero-sized pyo3_ffi type PyFrameObject
warning: ignoring zero-sized pyo3_ffi type PyGenObject
warning: ignoring zero-sized pyo3_ffi type PyInterpreterState
warning: ignoring zero-sized pyo3_ffi type PyLongObject
warning: ignoring zero-sized pyo3_ffi type PyThreadState
warning: ignoring zero-sized pyo3_ffi type _PyInterpreterFrame
nox > Session ffi-check was successful in 41 seconds.
Actually I guess the errors are for a freethreaded Python.
Based on what @alex has linked, probably we have the wrong definition of PyObject_HEAD_INIT, which ffi-check currently doesn't catch.
Definition of PyObject and PyObject_HEAD_INIT on 3.15t:
// Objects that are not owned by any thread use a thread id (tid) of zero.
// This includes both immortal objects and objects whose reference count
// fields have been merged.
#define _Py_UNOWNED_TID 0
struct _object {
// ob_tid stores the thread id (or zero). It is also used by the GC and the
// trashcan mechanism as a linked list pointer and by the GC to store the
// computed "gc_refs" refcount.
_Py_ALIGNED_DEF(_PyObject_MIN_ALIGNMENT, uintptr_t) ob_tid;
uint16_t ob_flags;
PyMutex ob_mutex; // per-object lock
uint8_t ob_gc_bits; // gc-related state
uint32_t ob_ref_local; // local reference count
Py_ssize_t ob_ref_shared; // shared (atomic) reference count
PyTypeObject *ob_type;
};
#if defined(Py_GIL_DISABLED)
#define PyObject_HEAD_INIT(type) \
{ \
0, \
_Py_STATICALLY_ALLOCATED_FLAG, \
{ 0 }, \
0, \
_Py_IMMORTAL_REFCNT_LOCAL, \
0, \
(type), \
},
#else
From pymacro.h
// _Py_ALIGNED_DEF(N, T): Define a variable/member with increased alignment
//
// `N`: the desired minimum alignment, an integer literal, number of bytes
// `T`: the type of the defined variable
// (or a type with at least the defined variable's alignment)
//
// May not be used on a struct definition.
//
// Standards/compiler support for `alignas` alternatives:
// - `alignas` is a keyword in C23 and C++11.
// - `_Alignas` is a keyword in C11
// - GCC & clang has __attribute__((aligned))
// (use that for older standards in pedantic mode)
// - MSVC has __declspec(align)
// - `_Alignas` is common C compiler extension
// Older compilers may name `alignas` differently; to allow compilation on such
// unsupported platforms, we don't redefine _Py_ALIGNED_DEF if it's already
// defined. Note that defining it wrong (including defining it to nothing) will
// cause ABI incompatibilities.
//
// Behavior of `alignas` alternatives:
// - `alignas` & `_Alignas`:
// - Can be used multiple times; the greatest alignment applies.
// - It is an *error* if the combined effect of all `alignas` modifiers would
// decrease the alignment.
// - Takes types or numbers.
// - May not be used on a struct definition, unless also defining a variable.
// - `__declspec(align)`:
// - Has no effect if it would decrease alignment.
// - Only takes an integer literal.
// - May be used on struct or variable definitions.
// However, when defining both the struct and the variable at once,
// `declspec(aligned)` causes compiler warning 5274 and possible ABI
// incompatibility.
// - ` __attribute__((aligned))`:
// - Has no effect if it would decrease alignment.
// - Takes types or numbers
// - May be used on struct or variable definitions.
#ifndef _Py_ALIGNED_DEF
# ifdef __cplusplus
# if __cplusplus >= 201103L
# define _Py_ALIGNED_DEF(N, T) alignas(N) alignas(T) T
# elif defined(__GNUC__) || defined(__clang__)
# define _Py_ALIGNED_DEF(N, T) __attribute__((aligned(N))) T
# elif defined(_MSC_VER)
# define _Py_ALIGNED_DEF(N, T) __declspec(align(N)) T
# else
# define _Py_ALIGNED_DEF(N, T) alignas(N) alignas(T) T
# endif
# elif defined(_MSC_VER)
# define _Py_ALIGNED_DEF(N, T) __declspec(align(N)) T
# elif defined(__STDC_VERSION__) && __STDC_VERSION__ >= 202311L
# define _Py_ALIGNED_DEF(N, T) alignas(N) alignas(T) T
# elif defined(__STDC_VERSION__) && __STDC_VERSION__ >= 201112L
# define _Py_ALIGNED_DEF(N, T) _Alignas(N) _Alignas(T) T
# elif (defined(__GNUC__) || defined(__clang__))
# define _Py_ALIGNED_DEF(N, T) __attribute__((aligned(N))) T
# else
# define _Py_ALIGNED_DEF(N, T) _Alignas(N) _Alignas(T) T
# endif
#endif
@ngoldbaum mind taking a peek?
Currently I've got my plate full trying to get Python 3.14 support working in packages. We'll probably want to start looking at 3.15 eventually, but probably not until next year.
