cpython icon indicating copy to clipboard operation
cpython copied to clipboard

Running Cython fails with 3.14.0a1 - a list attribute gets changed to `None`

Open da-woods opened this issue 1 year ago • 2 comments

Bug report

Bug description:

To be clear - this issue is just about running the Cython itself (i.e. pure Python code). It is not about compiled extension modules.

To reproduce

  • Checkout Cython from git: https://github.com/cython/cython.git
  • git checkout daed3bce0bf0c6fb9012170cb479f64e8b9532cd (probably not important, but let's make sure we're all definitely starting from the same point).
  • python3.14 cython.py Cython/Compiler/Parsing.py - this runs Cython on one of its own files

You get an output that ends with

File 'ExprNodes.py', line 8643, in analyse_types: TupleNode(Parsing.py:750:34,
    is_sequence_constructor = 1,
    result_is_used = True,
    use_managed_ref = True)

Compiler crash traceback from this point on:
  File "/home/dave/Documents/programming/cython2/Cython/Compiler/ExprNodes.py", line 8643, in analyse_types
    if len(self.args) == 0:
       ~~~^^^^^^^^^^^
TypeError: object of type 'NoneType' has no len()

To try to debug it some more I add a constructor to TupleNode (in Cython/Compiler/ExprNodes.py at line 8627) with a breakpoint:

def __init__(self, pos, **kw):
        if "args" in kw and kw['args'] is None:
            breakpoint()
        super().__init__(pos, **kw)
> <fullpath>/Cython/Compiler/ExprNodes.py(8629)__init__()
-> breakpoint()
(Pdb) print(kw)
{'args': None}
(Pdb) up
> <fullpath>/Cython/Compiler/ExprNodes.py(6123)analyse_types()
-> self.arg_tuple = TupleNode(self.pos, args = self.args)
(Pdb) print(self.args)
[<Cython.Compiler.ExprNodes.NameNode object at 0x7fb636188c80>]
(Pdb) 

So the the constructor call to TupleNode args is None. But in the function it's being called from it's a list containing a NameNode (which is what I think it should be). That's as far as I've got with debugging.

This has apparently been bisected to https://github.com/python/cpython/pull/122620 (but not by me).

CPython versions tested on:

3.14

Operating systems tested on:

Linux

Linked PRs

  • gh-125876

da-woods avatar Oct 23 '24 06:10 da-woods

My guess is that https://github.com/python/cpython/pull/122620 is to blame. Specifically removing this deopt https://github.com/python/cpython/pull/122620/files#diff-729a985b0cb8b431cb291f1edb561bbbfea22e3f8c262451cd83328a0936a342L2221

If a value is NULL, then the usual invariants on keys might not apply to that key.

Suppose we had a dict {'a': 'b'} with a table looking like this:

key value
"a" "b"
"a" NULL

I'm not sure how such a table would come into being, but it might be possible through a convoluted sequence of additions and deletions

If a STORE_ATTR_WITH_HINT had index=1 and key="a" then it would change the table to

key value
"a" "b"
"a" NEW_VALUE

which definitely isn't legal.

markshannon avatar Oct 23 '24 10:10 markshannon

I tried to compile numpy from @markshannon fix branch and the Cython compilation suddenly worked. Thank you!

leftys avatar Oct 23 '24 19:10 leftys

Closing as the (reported) fix went into 3.14.0a2

scoder avatar Dec 05 '24 08:12 scoder