black
black copied to clipboard
psf
Parsing the with statement with tuple argument and type comment failed
Describe the bug
The following code can not be parsed/formatted by black:
with (name_5, something): # type: ignoresome text
pass
black reported the following error:
> black -l 100 -t py312 bug.py
error: cannot format bug.py: INTERNAL ERROR: Black 25.1.1.dev21+g944a38e.d20250317 on Python (CPython) 3
.12.6 produced code that is not equivalent to the source. Please report a bug on https://github.com/psf
/black/issues. This diff might be helpful: /tmp/blk_pgrn0rje.log
Oh no! 💥 💔 💥
1 file failed to reformat.
the reported diff in /tmp/blk_pgrn0rje.log is:
--- src
+++ dst
@@ -5,30 +5,29 @@
Pass(
) # /Pass
items=
withitem(
context_expr=
- Tuple(
+ Name(
ctx=
Load(
) # /Load
- elts=
- Name(
- ctx=
- Load(
- ) # /Load
- id=
- 'name_5', # str
- ) # /Name
- Name(
- ctx=
- Load(
- ) # /Load
- id=
- 'something', # str
- ) # /Name
- ) # /Tuple
+ id=
+ 'name_5', # str
+ ) # /Name
+ optional_vars=
+ None, # NoneType
+ ) # /withitem
+ withitem(
+ context_expr=
+ Name(
+ ctx=
+ Load(
+ ) # /Load
+ id=
+ 'something', # str
+ ) # /Name
optional_vars=
None, # NoneType
) # /withitem
type_comment=
'ignoresome text', # str
but it can be parsed by cpython:
from ast import parse
parse(
'with (name_5, something): # type: ignoresome text\n'
' pass\n'
)
The code can be formatted with black -l 100 -t py312 bug.py --fast:
with name_5, something: # type: ignoresome text
pass
Environment
- Black's version: current main (6144c46c6a6960aeaf62484d3d8cbafedf0092f3)
- OS and Python version: Linux/Python 3.12.6 (main, Sep 9 2024, 22:11:19) [Clang 18.1.8 ]
Additional context
The bug was found by pysource-codegen (see #3908) The above problem description was created from a script, let me know if you think it can be improved.
Happy to report that the tool infact hit the minimal reproduction.
It can be changed cosmetically slightly:
with (x, y): # type:
pass
But I'm very impressed.
And I'm happy to report that this is most likely the last bug the tool has found.
I see the problem now, it is a deviation from expectation from CPython itself (at least from what I think)
In case of a type COMMENT, we get DIFFERENT ASTs, with and without the brackets:
>>> import ast
>>> x = '''
... with (x, y): # type: int
... pass
... '''
>>> y = '''
... with x, y: # type: int
... pass
... '''
>>> print(ast.dump(ast.parse(x, type_comments=True), indent=2))
Module(
body=[
With(
items=[
withitem(
context_expr=Tuple(
elts=[
Name(id='x', ctx=Load()),
Name(id='y', ctx=Load())],
ctx=Load()))],
body=[
Pass()],
type_comment='int')],
type_ignores=[])
>>> print(ast.dump(ast.parse(y, type_comments=True), indent=2))
Module(
body=[
With(
items=[
withitem(
context_expr=Name(id='x', ctx=Load())),
withitem(
context_expr=Name(id='y', ctx=Load()))],
body=[
Pass()],
type_comment='int')],
type_ignores=[])
In case of type IGNORE, we get IDENTICAL ASTs with or without bracket:
>>> x = '''
... with (x, y): # type: ignore
... pass
... '''
>>> y = '''
... with x, y: # type: ignore
... pass
... '''
>>> print(ast.dump(ast.parse(x, type_comments=True), indent=2))
Module(
body=[
With(
items=[
withitem(
context_expr=Name(id='x', ctx=Load())),
withitem(
context_expr=Name(id='y', ctx=Load()))],
body=[
Pass()])],
type_ignores=[
TypeIgnore(lineno=2, tag='')])
>>> print(ast.dump(ast.parse(y, type_comments=True), indent=2))
Module(
body=[
With(
items=[
withitem(
context_expr=Name(id='x', ctx=Load())),
withitem(
context_expr=Name(id='y', ctx=Load()))],
body=[
Pass()])],
type_ignores=[
TypeIgnore(lineno=2, tag='')])
Honestly this feels like a CPython bug, type comment on a line doesn't change semantic meaning and therefore should not change the AST.
Update on this: This was fixed in Python 3.13, and the bug doesn't reproduce on Python 3.13 either. I think it's safe to close this, but I'll let @JelleZijlstra decide.
I think it would be ok to search the issues with pysource-codegen only with python 3.13, or to build some sort of white-list if needed.
