isinstance return different result with sys.setprofile

[lonsdale8734]

python: 3.8.5

import sys
from typing import TypeVar


def run():
    sys.modules.pop("typing_extensions", None)
    from typing_extensions import ParamSpec
    return isinstance(ParamSpec("P"), TypeVar)


def trace_call(frame, event, arg):
   return trace_call


if __name__ == "__main__":
    print(run()) # True

    sys.setprofile(trace_call)
    print(run()) # False??

    sys.setprofile(None)
    print(run()) #  True

[srittau]

FWIW, I can reproduce this with Python 3.8.18 and 3.9.18, but not with 3.10.13 and 3.11.6, where the typing version of ParamSpec is re-exported.

[AlexWaygood]

After some investigation at the PyCon sprints, we determined that this is caused by a CPython bug where __class__ is removed from the class dictionary in some -- but not all -- situations when a profiling function has been set. (We trigger this bug in typing_extensions due to the way we set __class__ = typing.TypeVar in our implementation of typing_extensions.ParamSpec on Python 3.8 and 3.9.)

The bug only exists on Python <=3.10. It can be demonstrated in the following reproducible example. We can see that instances of GlobalParamSpec in the following snippet are always instances of TypeVar, whether or not a profiling function has been set. However, instances of LocalParamSpec are not instances of TypeVar when a profiling function has been set -- but are otherwise -- because __class__ has been removed from the class dictionary of LocalParamSpec.

Minimal repro:

import sys

class TypeVar: pass

class GlobalParamSpec:
    __class__ = TypeVar
    def m(self):
        __class__

def run():
    class LocalParamSpec:
        __class__ = TypeVar
        def m(self):
            __class__

    print(
        "Global class: ",
        isinstance(GlobalParamSpec(), TypeVar),
        GlobalParamSpec.__dict__.get('__class__'),
        "Local class: ",
        isinstance(LocalParamSpec(), TypeVar),
        LocalParamSpec.__dict__.get('__class__'),
    )

def trace_call(frame, event, arg):
    # print(frame, event, arg)
    return trace_call

if __name__ == "__main__":
    print('Without profiling function')
    run()
    print()
    sys.setprofile(trace_call)
    print('With profiling function')
    run()
    print()
    sys.setprofile(None)
    print('Without profiling function again')
    run()

Output of the script, on Python 3.8, 3.9 and 3.10:

Without profiling function
Global class:  True <class '__main__.TypeVar'> Local class:  True <class '__main__.TypeVar'>

With profiling function
Global class:  True <class '__main__.TypeVar'> Local class:  False None

Without profiling function again
Global class:  True <class '__main__.TypeVar'> Local class:  True <class '__main__.TypeVar'>

Thanks to @brandtbucher and @carljm for helping debug this!

[AlexWaygood]

I bisected which commit during Python 3.11 development fixed the CPython bug (git bisect says the "first bad commit", but it's really the "first good commit"):

commit d7163bb35d1ed46bde9affcd4eb267dfd0b703dd (HEAD)
Author: Mark Shannon <[email protected]>
Date:   Fri Mar 25 12:57:50 2022 +0000

    bpo-42197: Don't create `f_locals` dictionary unless we actually need it. (GH-32055)
    
    * `PyFrame_FastToLocalsWithError` and `PyFrame_LocalsToFast` are no longer called during profile and tracing.
     (Contributed by Fabio Zadrozny)
    
    * Make accesses to a frame's `f_locals` safe from C code, not relying on calls to `PyFrame_FastToLocals` or `PyFrame_LocalsToFast`.
    
    * Document new `PyFrame_GetLocals` C-API function.

https://github.com/python/cpython/commit/d7163bb35d1ed46bde9affcd4eb267dfd0b703dd

[AlexWaygood]

https://github.com/python/cpython/commit/d7163bb35d1ed46bde9affcd4eb267dfd0b703dd didn't really fix the underlying CPython bug. It actually just made f_locals lazy, so it means that the bug is no longer reproducible by a no-op trace function, as the no-op trace function never accesses f_locals. If you change the script to the following, so that f_locals is accessed by the trace function, the bug still reproduces on Python 3.11 and 3.12:

import sys

class A: pass

class GlobalB:
    __class__ = A
    def m(self):
        __class__

def run():
    class LocalB:
        __class__ = A
        def m(self):
            __class__

    print(
        "Global class: ",
        isinstance(GlobalB(), A),
        GlobalB.__dict__.get('__class__'),
        "Local class: ",
        isinstance(LocalB(), A),
        LocalB.__dict__.get('__class__'),
    )

def trace_call(frame, event, arg):
    frame.f_locals
    return trace_call

if __name__ == "__main__":
    print('Without profiling function')
    run()
    print()
    sys.setprofile(trace_call)
    print('With profiling function')
    run()
    print()
    sys.setprofile(None)
    print('Without profiling function again')
    run()

The bug does not reproduce on 3.13.0b1 or the CPython main branch... but it was only fixed a few weeks ago! It was fixed (unsurprisingly) in https://github.com/python/cpython/commit/b034f14a4b6e9197d3926046721b8b4b4b4f5b3d.