pygments icon indicating copy to clipboard operation
pygments copied to clipboard

'str' object has no attribute 'parent'

Open AnthonySkorski opened this issue 3 years ago • 7 comments

pgcli is broken with version 2.11.2 of pygments, see https://github.com/dbcli/pgcli/issues/1303

I did not logged the pgcli issue but I have the exact same error and downgrading pygments to 2.11.1 also solves my issue.

AnthonySkorski avatar Jan 07 '22 13:01 AnthonySkorski

Which code did you highlight to trigger this?

Anteru avatar Jan 07 '22 14:01 Anteru

I tried to reproduce this and there's no token type Output.TableSeparator in Pygments, so this must be a token type that pgcli is generating. I suspect pgcli returns a string for a token instead of a native Pygments token type and then tries to highlight it using Pygments. Previously, for rather dubious reasons, Pygments used ttype[-1] to walk up to the parent in the Terminal256 lexer, and ttype.parent everywhere else. In 2.11.2, Terminal256 uses ttype.parent now, which fixes several highlighting issues, but seems to collide with either what pgcli or prompt_toolkit is doing. I suspect it's exploiting some internals which happened to work with Terminal256 only.

Anteru avatar Jan 07 '22 14:01 Anteru

since it occurs when trying to execute a query with pgcli (a CLI postgres client) and the query itself seems to be correctly highlighted, I assume that the text causing the error is a tabular representation of the query result. Something like the following:

+---------+---------+
| columnA | columnB |
|---------+---------|
| valueA1 | valueB1 |
| valueA2 | valueB2 |
+---------+---------+

AnthonySkorski avatar Jan 07 '22 14:01 AnthonySkorski

If I had to guess, +------+---------+ is Token.Output.TableSeparator, but it's missing from this dictionary: https://github.com/dbcli/pgcli/blob/master/pgcli/pgstyle.py#L13 and thus the whole thing falls apart.

Anteru avatar Jan 07 '22 14:01 Anteru

Ok thanks for the time spent on investigation. I will add a link to your conclusion in the pgcli issue, hoping it will help.

AnthonySkorski avatar Jan 07 '22 14:01 AnthonySkorski

Looking even further into this, the offending token comes from here: https://github.com/dbcli/cli_helpers/blob/master/cli_helpers/tabular_output/tabulate_adapter.py#L108 -- and the fix might be as simple as changing line 108 to = Token.Output.TableSeparator. What I think is happening is that the style in pgcli is missing that type, and the fallback fails as well.

Anteru avatar Jan 07 '22 15:01 Anteru

While this has been worked around by pinning the Pygments version, this is not a long-term fix for this issue. I don't see how this is a bug in Pygments though as the previous code just happened to not crash, but wasn't doing anything useful either (it was applying ttype[-1] on a string and walking that one back.) @birkenfeld Any thoughts? Otherwise I'm closing this and will open an issue that https://github.com/dbcli/cli_helpers/blob/2a7d3797dc4ee706a215aebc73f78284abf1eae4/cli_helpers/tabular_output/tabulate_adapter.py#L108 needs to get changed/fixed. I may be missing some history on Pygments though -- where string tokens types ever officially supported? I could add a shim as well which tries to build a token instance from a string by splitting it at the dots or something along those lines ...

Anteru avatar Jan 11 '22 21:01 Anteru