[BUG] linetrace=True interacts badly with VSCode debugger

[da-woods]

Describe the bug

If you create a module with linetrace=True and try to debug code that imports it then you get an assertion error. Note that this happens even if you step over the Python module.

Code to reproduce the behaviour:

# cython: linetrace=True

Just compile that with cythonize.py -if my_cython_module.pyx

Then a Python file:

print(1)
import my_cython_module
print(2)

Using VSCode, put a breakpoint on line 1 (print(1)). Run the module in debug mode. Step to line 2 (import ...). Step to line 3. Here you get a popup saying:

Exception has occurred: AssertionError
<code object _call_with_frames_removed at 0x798ad8e20570, file "<frozen importlib._bootstrap>", line 480> != <code object PyInit_my_cython_module at 0x798ad73d5b00, file "my_cython_module.pyx", line 1>
KeyError: (1, '<path>/my_cython_module.pyx', <code object PyInit_my_cython_module at 0x798ad73d5b00, file "my_cython_module.pyx", line 1>)

During handling of the above exception, another exception occurred:

  File "<path>/my_cython_module.pyx", line 1, in init my_cython_module
    # cython: linetrace=True
  File "<path>/debugimport.py", line 2, in <module>
    import my_cython_module
AssertionError: <code object _call_with_frames_removed at 0x798ad8e20570, file "<frozen importlib._bootstrap>", line 480> != <code object PyInit_my_cython_module at 0x798ad73d5b00, file "my_cython_module.pyx", line 1>

Expected behaviour

No assertion error?

OS

This is on Windows VSCode running Python in WSL. I suspect it's general though

Python version

3.13, probably 3.12

Cython version

Current master. Anything after https://github.com/cython/cython/commit/ed2b40be6cad736ab39cffa2503d816ef2c6741e

Additional context

The assertion error comes from https://github.com/fabioz/PyDev.Debugger/blob/3030fa9908187513bdd3e3dd93bfa897a95a6ece/_pydevd_sys_monitoring/_pydevd_sys_monitoring.py#L615

It's possible the issue may be on their side.

See https://github.com/rapidfuzz/RapidFuzz/issues/454

[da-woods]

See also https://github.com/fabioz/PyDev.Debugger/issues/311

[da-woods]

In principle we might be able to pre-disable the debugger tool in the monitoring state (on the basis that it's unlikely to ever be too useful with Cython).

That seems overly pessimistic though - it does prevent other people doing useful stuff with it in future

[da-woods]

Also https://github.com/microsoft/debugpy/issues/1757 - it'd a little unfair to try to blame PyDev.Debugger for this because Microsoft have vendored their library, and so most people will be hitting Microsoft's copy

[scoder]

A major goal of supporting sys.monitoring, from the PoV of Cython, was to get over the need for clumsily emulating frame state. Python interpreter frames are not part of the monitoring infrastructure, code objects are. Thus, it seems reasonable for a Python code debugger to check if the current Python frame refers to the monitored code object, but the effect of this check should not be a failing assertion. It's not invalid for code objects and Python frame state to diverge. A Python code debugger should simply ignore monitoring events for which there is no Python frame, not fail on them.

Regarding a solution or work-around, there are tens of thousands of Cython modules out there but only a hand full of Python debuggers. That hand full seems substantially easier to fix than masses of Python packages.

[da-woods]

I agree, although I'm not optimistic about getting debuggers fixed.