LibCST icon indicating copy to clipboard operation
LibCST copied to clipboard

AnnAssign allows invalid whitespace around equals

Open AustinScola opened this issue 4 years ago • 4 comments

I noticed that AnnAssign allows for whitespace which includes newlines before or after the equals without a line continuation character. Is this expected? It seems when you construct other nodes, they throw a CSTValidationError if the node would represent invalid Python code.

Here is an example where there is a newline before the equals sign:

from libcst import (AnnAssign, Annotation, AssignEqual, Name,
                    ParenthesizedWhitespace, TrailingWhitespace)

AnnAssign(
    target=Name(
        value='foo',
    ),
    annotation=Annotation(
        annotation=Name(
            value='Foo'
        ),
    ),
    value=Name(
        value='bar'
    ),
    equal=AssignEqual(
        whitespace_before=ParenthesizedWhitespace(
            first_line=TrailingWhitespace()
        ),
    )
)

And this would represent the following:

foo:Foo
=bar

AustinScola avatar May 21 '21 13:05 AustinScola

Yep, looks like this is a bug. This returns the broken source code

from libcst import AnnAssign, Annotation, AssignEqual, Name, ParenthesizedWhitespace, TrailingWhitespace, Module, SimpleStatementLine

Module(body=[
    SimpleStatementLine(body=[
        AnnAssign(
            target=Name(
                value='foo',
            ),
            annotation=Annotation(
                annotation=Name(
                    value='Foo'
                ),
            ),
            value=Name(
                value='bar'
            ),
            equal=AssignEqual(
                whitespace_before=ParenthesizedWhitespace(
                    first_line=TrailingWhitespace()
                ),
            )
        )
    ])  
]).code

zsol avatar May 21 '21 16:05 zsol

I'm not entirely convinced this is a libcst bug, or if it is, it's somewhere other than codegen.

The whitespace you have is valid in ParenthesizedWhitespace, but if there are no enclosing parens, it should be using SimpleWhitespace. How did you create this node?

See docstrings in https://github.com/Instagram/LibCST/blob/master/libcst/_nodes/whitespace.py

thatch avatar May 21 '21 19:05 thatch

I think maybe there is some confusion about my usage of libcst. I didn't create the node via parsing anything. Rather, I manually created it using code similar to the above. It is my understanding that upon initialization of a node libcst raises a CSTValidationError in circumstances like this where the node isn't representative of valid Python code? Please correct me if my expectations are wrong though.

It would appear to me that only SimpleWhitespace should be allowed in an AnnAssign and not ParenthesizedWhitespace?

AustinScola avatar May 21 '21 21:05 AustinScola

I spent a few minutes trying and have not managed to generate a parenthesized call where the whitespace before the : can contain newlines. I think you might be right, but we'll need to check the grammar to be sure.

thatch avatar May 21 '21 23:05 thatch