Better parsing strings that contain `{{`

[FMorschel]

Better parsing strings that contain double brackets ({{)

I've created this issue on the Python repo because, in the following code, the parsing of escaping brackets was not clear to me.

print(f'Explicit map {{1: 2}}.')

print(f'Explicit map keys {{1:2}.keys()}.')

In Python, having two following brackets {{ or }} is how you escape them and they appear on the string, so the above code has an error at the second print because it can't have only single closing brackets inside the print. You can run it by adding a space between the two opening brackets in the second print. I'll show the difference it makes in both vscode and PyCharm:

Screenshot from vscode without parsing double brackets

Screenshot from vscode spacing double brackets

As I've mentioned there, already, in PyCharm this colouring is already happening as I suggested. It should probably paint the double brackets the same colour as \n and other special escaping characters.

Screenshot from PyCharm parsing double brackets

Screenshot from PyCharm spacing double brackets

[debonte]

@FMorschel, thanks for the issue. Can you please clarify what you are requesting here?

The two VS Code screenshots look identical to me.

In the PyCharm screenshots, it's not clear to me what you mean by "with parsing brackets" and "without parsing brackets". Is there a config setting that you are changing?

[FMorschel]

I'm sorry that I was not clear enough from the start. I edited the OP. I hope it's clearer now. Also, I removed the boilerplate I brought from my previous issue mentioned there.

[debonte]

The TextMate scope for the escaped {{ is correct -- constant.character.escape.python, which is the same as \n.

The {{ is shown with two colors because VS Code's editor.bracketPairColorization.enabled config setting is enabled by default. If you set it to false, the brackets are colored as you expected:

But of course, disabling bracket pair colorization would be painful in other areas of your code. For example, seeing this: instead of this:

[debonte]

@hediet, the problem here is that escaped bracket characters within Python f-strings are colored by VS Code's "bracket pair colorization" feature.

Is this a VS Code bug, or is there something Pylance should be doing (ex. additional metadata somewhere?) to help VS Code understand that these chars are escaped?

[FMorschel]

I know of this setting, but I also know that if you look here in the Dart extension for vscode they have all of their brackets painted right even when they are inside strings. I'm not sure how they do it but if it helps, I actually came from an issue related where they are solving a small problem in that.

https://github.com/microsoft/vscode/issues/209435

[hediet]

ex. additional metadata somewhere?

Can you try the "Inspect Tokens" command in VS Code and check if f-strings are marked as string?

[RedCMD]

in the Dart extension

@FMorschel they use the scopename meta.embedded to reset the standardTokenType from string back to other

[FMorschel]

Thanks, RedCMD. Just in case anyone needs to see this to solve this issue, here is a bug/request (I'm not sure how to call that) and the quote from the issue I mentioned above explaining why its created:

The closest I've got without using meta.embedded is by listing alternating tokenTypes like this:

"tokenTypes": {
	"my-code": "other",
	"my-code string": "string",
	"my-code string my-code": "other",
	"my-code string my-code string": "string",
	"my-code string my-code string my-code": "other",
	"my-code string my-code string my-code string": "string"
}

Obviously this is silly, because I can't do this forever. I've opened #210053 about this because it feels like it should be possible to do this without having to use meta.embedded.

For now, I'll probably just go with the above to some reasonable number of levels though (after some more testing).

[debonte]

Can you try the "Inspect Tokens" command in VS Code and check if f-strings are marked as string?

@hediet, here's what I see for the string text (ex. "Explicit map keys "):

And here's what I see for the {{: